TSP

@Idea-ahead I sent you a message on my modifications done to the Item.php file for 4.7.17. I expect one of those two methods is the culprit. Please note as I said that my customizations was made for 4.7.17, and I haven't reviewed the changes made to that file in newer versions 4.7.18 and 4.7.19 yet, so it may not be directly compatible.

Hi, We are currently considering switching from MariaDB to MySQL 8. However, we would prefer if we would then be able to go straight to MySQL 8.4. I see that in the requirements.json you list MySQL 8.0 as recommended. Are there any known issues with choosing MySQL 8.4 with Invision Community version 4.7? And what are the plans for supporting newer versions of MySQL?

We noticed similar issues going from 4.7.13 to 4.7.16. I was in contact with @Matt back then. In our case we ended up reverting to an the code in getItemsWithPermission from an earlier version where we hadn't had that issue with getItemsWithPermission method. We also got another issue upgrading to 4.7.17. In that case we ended up reverting some code in _comments() Using Mariadb 10.5. But we'll upgrade to either a newer version of MariaDB or Mysql 8.0 or Mysql 8.4 soon-ish, so we hopefully can remove these customizations.

I personally don't see a reason for such a confirmation box in that case, it's easy enough to just change the status again if you did a mistake. Guess I'll try to make a hook to remove it if no notifications are set up.

Cool, but if you don't want to set up notifications or use that system, it seems unneccessary to get a popup like this each time you want to mark a report complete or rejected. Hope this is a bug that can be corrected.

A long time ago I think I remember it being mentioned, when you first said display name logins would be removed, that you would have some system in place before that to ensure that people were aware of their registered email address, and be sure to keep it updated. Will something like that come to the 4.x line?

We have some. I've provided more details in a reply I just made to the ticket.

Steps to reproduce: Create a test topic in a forum where guests does NOT have permission to post new topics Reply with a test account, or temporarily allow guests to post topics and replies. If you replied with a test account: delete the test account so the reply becomes a guest post If you posted as a guest: remember to reset the forum permissions to once again disallow guest topics/replies As a moderator: select the reply and split it into a new topic in the same forum with your moderator tools You should be redirected to the new topic created by splitting the posts Instead you'll get an error page: "You do not have permission to view this topic." (Error code: 2F173/K) The reason is that the new topic is saved to the database with the approved column set to -3. Which in system/Content/Content.php is documented as: "@li -3 is a post made by a guest using the "post before register" feature" Please note that this will happen even if the system "Post before register" is disabled. It shouldn't happen in either cases. Also; since there is no way to get a list of topics and replies hidden for this reason, it's not possible to save the topic from the interface, you'll have to manually fix it in the database. I think this happens because of the following code for the function createItem in system/Content/Item.php. I'm pretty sure you'll have to fix the logic for both the approved and hidden switch. <?php case 'approved': if ( $hidden === NULL ) { if ( !$author->member_id and $container and !$container->can( 'add', $author, FALSE ) ) { $val = -3; } else { $val = static::moderateNewItems( $author, $container ) ? 0 : 1; } } else { $val = \intval( !$hidden ); } break; case 'hidden': if ( $hidden === NULL ) { if ( !$author->member_id and $container and !$container->can( 'add', $author, FALSE ) ) { $val = -3; } else { $val = static::moderateNewItems( $author, $container ) ? 1 : 0; } } else { $val = \intval( $hidden ); } break;

@Stuart Silvester does my reply about database version help at all? Is this a known issue that is being worked on?

Server version: 10.5.24-MariaDB-1:10.5.24+maria~ubu2004-log mariadb.org binary distribution

Thanks for the feedback 🙂 I presume you have 4.7.16 on your test server, and that your production server is still running an earlier version, which version? Between versions 4.7.13 and 4.7.16 I see a bunch of changes in system/Content/Item.php. However, I don't have a changelog for the in between versions. Do anyone know which versions these changes were introduced? I suspect some of the changes has to do with the change in database server utilization... Here are the diff for the changes in the getItemsWithPermission-method in the file system/Content/Item.php: @@ -3159,11 +3161,16 @@ abstract class _Item extends \IPS\Content { $containerWhere = array_merge( $containerWhere, $value ); unset( $where[ $key ] ); + + /* $containerWhere is used for exclusion purposes now, + so we leave this as part of the where condition as well. */ + $where = array_merge( $where, $value ); } } } /* Exclude hidden items */ + $includeAdditionalApprovalClauses = true; if( $includeHiddenItems === \IPS\Content\Hideable::FILTER_AUTOMATIC ) { $containersTheUserCanViewHiddenItemsIn = static::canViewHiddenItemsContainers( $member ); @@ -3199,6 +3206,8 @@ abstract class _Item extends \IPS\Content $col = static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['hidden']; $where[] = array( "{$col}=1" ); } + + $includeAdditionalApprovalClauses = false; } elseif ( \in_array( 'IPS\Content\Hideable', class_implements( \get_called_class() ) ) and $includeHiddenItems !== \IPS\Content\Hideable::FILTER_SHOW_HIDDEN ) { @@ -3226,6 +3235,8 @@ abstract class _Item extends \IPS\Content { $where[] = array( "{$col}=1" ); } + + $includeAdditionalApprovalClauses = false; } elseif ( isset( static::$databaseColumnMap['hidden'] ) ) { @@ -3247,6 +3258,8 @@ abstract class _Item extends \IPS\Content { $where[] = array( "{$col}=0" ); } + + $includeAdditionalApprovalClauses = false; } } else @@ -3274,17 +3287,21 @@ abstract class _Item extends \IPS\Content } } - /* No matter if we can or cannot view hidden items, we do not want these to show: -2 is queued for deletion and -3 is posted before register */ - if ( isset( static::$databaseColumnMap['hidden'] ) ) - { - $col = static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['hidden']; - $where[] = array( "{$col}!=-2 AND {$col} !=-3" ); - } - else if ( isset( static::$databaseColumnMap['approved'] ) ) - { - $col = static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['approved']; - $where[] = array( "{$col}!=-2 AND {$col}!=-3" ); - } + /* This only makes sense if we have not already filtered by a single value */ + if( $includeAdditionalApprovalClauses ) + { + /* No matter if we can or cannot view hidden items, we do not want these to show: -2 is queued for deletion and -3 is posted before register */ + if ( isset( static::$databaseColumnMap['hidden'] ) ) + { + $col = static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['hidden']; + $where[] = array( "{$col}!=-2 AND {$col} !=-3" ); + } + else if ( isset( static::$databaseColumnMap['approved'] ) ) + { + $col = static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['approved']; + $where[] = array( "{$col}!=-2 AND {$col}!=-3" ); + } + } /* Future items? */ if ( \in_array( 'IPS\Content\FuturePublishing', class_implements( \get_called_class() ) ) ) @@ -3321,18 +3338,28 @@ abstract class _Item extends \IPS\Content $categories = array(); $lookupKey = md5( $containerClass::$permApp . $containerClass::$permType . $permissionKey . json_encode( $member->groups ) ); + /* EME: Switch to use exclusion instead of inclusion */ if( !isset( static::$permissionSelect[ $lookupKey ] ) ) { static::$permissionSelect[ $lookupKey ] = array(); - $permQuery = \IPS\Db::i()->select( 'perm_type_id', 'core_permission_index', array( "core_permission_index.app='" . $containerClass::$permApp . "' AND core_permission_index.perm_type='" . $containerClass::$permType . "' AND (" . \IPS\Db::i()->findInSet( 'perm_' . $containerClass::$permissionMap[ $permissionKey ], $member->permissionArray() ) . ' OR ' . 'perm_' . $containerClass::$permissionMap[ $permissionKey ] . "='*' )" ) ); + + $permQueryJoinContainer = (bool) ( \count( $containerWhere ) ); + $clubWhere = ""; /* If we cannot access clubs, skip them */ if ( \IPS\IPS::classUsesTrait( $containerClass, 'IPS\Content\ClubContainer' ) AND !$member->canAccessModule( \IPS\Application\Module::get( 'core', 'clubs', 'front' ) ) ) { - $containerWhere[] = array( $containerClass::$databaseTable . '.' . $containerClass::$databasePrefix . $containerClass::clubIdColumn() . ' IS NULL' ); + $permQueryWhere = array( "core_permission_index.app='" . $containerClass::$permApp . "' AND core_permission_index.perm_type='" . $containerClass::$permType . "' AND (" . $containerClass::$databaseTable . '.' . $containerClass::$databasePrefix . $containerClass::clubIdColumn() . " IS NOT NULL OR !(" . \IPS\Db::i()->findInSet( 'perm_' . $containerClass::$permissionMap[ $permissionKey ], $member->permissionArray() ) . ' OR ' . 'perm_' . $containerClass::$permissionMap[ $permissionKey ] . "='*' ))" ); + $permQueryJoinContainer = true; } + else + { + $permQueryWhere = array( "core_permission_index.app='" . $containerClass::$permApp . "' AND core_permission_index.perm_type='" . $containerClass::$permType . "' AND !(" . \IPS\Db::i()->findInSet( 'perm_' . $containerClass::$permissionMap[ $permissionKey ], $member->permissionArray() ) . ' OR ' . 'perm_' . $containerClass::$permissionMap[ $permissionKey ] . "='*' )" ); + } + + $permQuery = \IPS\Db::i()->select( 'perm_type_id', 'core_permission_index', $permQueryWhere ); - if ( \count( $containerWhere ) ) + if( $permQueryJoinContainer ) { $permQuery->join( $containerClass::$databaseTable, array_merge( $containerWhere, array( 'core_permission_index.perm_type_id=' . $containerClass::$databaseTable . '.' . $containerClass::$databasePrefix . $containerClass::$databaseColumnId ) ), 'STRAIGHT_JOIN' ); } @@ -3347,11 +3374,7 @@ abstract class _Item extends \IPS\Content if( \count( $categories ) ) { - $where[] = array( static::$databaseTable . "." . static::$databasePrefix . static::$databaseColumnMap['container'] . ' IN(' . implode( ',', $categories ) . ')' ); - } - else - { - $where[] = array( static::$databaseTable . "." . static::$databasePrefix . static::$databaseColumnMap['container'] . '=0' ); + $where[] = array( static::$databaseTable . "." . static::$databasePrefix . static::$databaseColumnMap['container'] . ' NOT IN (' . implode( ',', $categories ) . ')' ); } } @@ -3363,8 +3386,10 @@ abstract class _Item extends \IPS\Content $select = \IPS\Db::i()->select( 'COUNT(*) as cnt', static::$databaseTable, $where, NULL, NULL, $groupBy, NULL, $queryFlags ); if ( $joinContainer AND isset( static::$containerNodeClass ) ) { + /* EME: Removed the $containerWhere from the join because it is now in the where clause. + We are now using $containerWhere to exclude forums that shouldn't be visible. */ $containerClass = static::$containerNodeClass; - $select->join( $containerClass::$databaseTable, array_merge( $containerWhere, array( static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['container'] . '=' . $containerClass::$databaseTable . '.' . $containerClass::$databasePrefix . $containerClass::$databaseColumnId ) ) ); + $select->join( $containerClass::$databaseTable, array( static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['container'] . '=' . $containerClass::$databaseTable . '.' . $containerClass::$databasePrefix . $containerClass::$databaseColumnId ) ); } if ( $joinComments ) { Could someone at Invision answer whether it would cause any issues to temporarily revert this method to the previous that worked for us? Or is there a lot of changes in that method dependent on other changes done elsewhere in the same or later version, so I would have to hunt down a lot of other stuff?

I upgraded from version 4.7.13 to 4.7.16 on 19th April, and we have since been having trouble with both reduced performance and more downtime. Our server host has looked into this and reported back a major increase in slow queries, major increase in CPU usage for the database server overall and less usage of MySQL Sorts. It seems this increase can be tracked down to methods using IPS\Content\_Item::getItemsWithPermission:317. We have a custom controller and a frontpage widget utilizing this method. Anyone else experienced this, or have any tips on what could be going on? Hope Invision could be interested in lending a hand investigating this too Best regards, Preben

I assume any tables installed by the plugin will remain in place? And you are just speaking about the table listing the plugins?

Yeah, I have to say I'm also worried about ad placement. We need a lot of control over how and where they are included. For example we have some ads which should display as a "horseshoe" around the content. To achieve this currently we have our own wrapper class around the necessary elements etc. Also, from looking in the video from Ehren there is a lot of settings to control and clicking into all sorts of menus with settings. Will there be some way to control it by editing a json file instead for example? That would make it easier and faster for many of us frontend and backend programmers to copy theme settings between different installations etc. Essentially what I'm asking for would be a "source mode" for the theme editor settings view on the frontend.

A few questions at the top of my mind, I'll probably have some later too. Currently I've made a shared core theme where all our customizations for five different forum installations are. So the HTML, Javascript and CSS is kept the same, but some different output, logic and elements are displayed or utilized based on logic that depends on the value in custom theme variables. I assume there is still some tool to export and import/upgrade custom templates and template hooks? Having to log in to each admin panel and define all these custom templates and hooks would be a pain. Is custom theme variables still a thing and accessible within these templates? And will theme variables that have been modified on the installation be reset or kept upon upgrade/import of a theme? Will the same variables and functions we have available to us when making a template hook today be available to us from these new custom templates and hooks? I realize the names of methods, how you may invoke them, template logic etc., will be different, but my point is; will all the variables and functions that can be used in the template we hook into be available to us?