TSP

I’m not logged in inkognito. It worked in the previous version to read the rss feed URL when not logged in, and my understanding, given the RSS link contains a key-element, is that it’s meant to work when not authenticated, given the key: https://invisioncommunity.com/discover/66.xml/?&member=135437&key=46f... (I have not pasted the full key here…)

@Idea-ahead I sent you a message on my modifications done to the Item.php file for 4.7.17. I expect one of those two methods is the culprit. Please note as I said that my customizations was made for 4.7.17, and I haven't reviewed the changes made to that file in newer versions 4.7.18 and 4.7.19 yet, so it may not be directly compatible.

Hi, We are currently considering switching from MariaDB to MySQL 8. However, we would prefer if we would then be able to go straight to MySQL 8.4. I see that in the requirements.json you list MySQL 8.0 as recommended. Are there any known issues with choosing MySQL 8.4 with Invision Community version 4.7? And what are the plans for supporting newer versions of MySQL?

We noticed similar issues going from 4.7.13 to 4.7.16. I was in contact with @Matt back then. In our case we ended up reverting to an the code in getItemsWithPermission from an earlier version where we hadn't had that issue with getItemsWithPermission method. We also got another issue upgrading to 4.7.17. In that case we ended up reverting some code in _comments() Using Mariadb 10.5. But we'll upgrade to either a newer version of MariaDB or Mysql 8.0 or Mysql 8.4 soon-ish, so we hopefully can remove these customizations.

I personally don't see a reason for such a confirmation box in that case, it's easy enough to just change the status again if you did a mistake. Guess I'll try to make a hook to remove it if no notifications are set up.

Cool, but if you don't want to set up notifications or use that system, it seems unneccessary to get a popup like this each time you want to mark a report complete or rejected. Hope this is a bug that can be corrected.

A long time ago I think I remember it being mentioned, when you first said display name logins would be removed, that you would have some system in place before that to ensure that people were aware of their registered email address, and be sure to keep it updated. Will something like that come to the 4.x line?

We have some. I've provided more details in a reply I just made to the ticket.

Steps to reproduce: Create a test topic in a forum where guests does NOT have permission to post new topics Reply with a test account, or temporarily allow guests to post topics and replies. If you replied with a test account: delete the test account so the reply becomes a guest post If you posted as a guest: remember to reset the forum permissions to once again disallow guest topics/replies As a moderator: select the reply and split it into a new topic in the same forum with your moderator tools You should be redirected to the new topic created by splitting the posts Instead you'll get an error page: "You do not have permission to view this topic." (Error code: 2F173/K) The reason is that the new topic is saved to the database with the approved column set to -3. Which in system/Content/Content.php is documented as: "@li -3 is a post made by a guest using the "post before register" feature" Please note that this will happen even if the system "Post before register" is disabled. It shouldn't happen in either cases. Also; since there is no way to get a list of topics and replies hidden for this reason, it's not possible to save the topic from the interface, you'll have to manually fix it in the database. I think this happens because of the following code for the function createItem in system/Content/Item.php. I'm pretty sure you'll have to fix the logic for both the approved and hidden switch. <?php case 'approved': if ( $hidden === NULL ) { if ( !$author->member_id and $container and !$container->can( 'add', $author, FALSE ) ) { $val = -3; } else { $val = static::moderateNewItems( $author, $container ) ? 0 : 1; } } else { $val = \intval( !$hidden ); } break; case 'hidden': if ( $hidden === NULL ) { if ( !$author->member_id and $container and !$container->can( 'add', $author, FALSE ) ) { $val = -3; } else { $val = static::moderateNewItems( $author, $container ) ? 1 : 0; } } else { $val = \intval( $hidden ); } break;

@Stuart Silvester does my reply about database version help at all? Is this a known issue that is being worked on?

Server version: 10.5.24-MariaDB-1:10.5.24+maria~ubu2004-log mariadb.org binary distribution

Thanks for the feedback 🙂 I presume you have 4.7.16 on your test server, and that your production server is still running an earlier version, which version? Between versions 4.7.13 and 4.7.16 I see a bunch of changes in system/Content/Item.php. However, I don't have a changelog for the in between versions. Do anyone know which versions these changes were introduced? I suspect some of the changes has to do with the change in database server utilization... Here are the diff for the changes in the getItemsWithPermission-method in the file system/Content/Item.php: @@ -3159,11 +3161,16 @@ abstract class _Item extends \IPS\Content { $containerWhere = array_merge( $containerWhere, $value ); unset( $where[ $key ] ); + + /* $containerWhere is used for exclusion purposes now, + so we leave this as part of the where condition as well. */ + $where = array_merge( $where, $value ); } } } /* Exclude hidden items */ + $includeAdditionalApprovalClauses = true; if( $includeHiddenItems === \IPS\Content\Hideable::FILTER_AUTOMATIC ) { $containersTheUserCanViewHiddenItemsIn = static::canViewHiddenItemsContainers( $member ); @@ -3199,6 +3206,8 @@ abstract class _Item extends \IPS\Content $col = static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['hidden']; $where[] = array( "{$col}=1" ); } + + $includeAdditionalApprovalClauses = false; } elseif ( \in_array( 'IPS\Content\Hideable', class_implements( \get_called_class() ) ) and $includeHiddenItems !== \IPS\Content\Hideable::FILTER_SHOW_HIDDEN ) { @@ -3226,6 +3235,8 @@ abstract class _Item extends \IPS\Content { $where[] = array( "{$col}=1" ); } + + $includeAdditionalApprovalClauses = false; } elseif ( isset( static::$databaseColumnMap['hidden'] ) ) { @@ -3247,6 +3258,8 @@ abstract class _Item extends \IPS\Content { $where[] = array( "{$col}=0" ); } + + $includeAdditionalApprovalClauses = false; } } else @@ -3274,17 +3287,21 @@ abstract class _Item extends \IPS\Content } } - /* No matter if we can or cannot view hidden items, we do not want these to show: -2 is queued for deletion and -3 is posted before register */ - if ( isset( static::$databaseColumnMap['hidden'] ) ) - { - $col = static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['hidden']; - $where[] = array( "{$col}!=-2 AND {$col} !=-3" ); - } - else if ( isset( static::$databaseColumnMap['approved'] ) ) - { - $col = static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['approved']; - $where[] = array( "{$col}!=-2 AND {$col}!=-3" ); - } + /* This only makes sense if we have not already filtered by a single value */ + if( $includeAdditionalApprovalClauses ) + { + /* No matter if we can or cannot view hidden items, we do not want these to show: -2 is queued for deletion and -3 is posted before register */ + if ( isset( static::$databaseColumnMap['hidden'] ) ) + { + $col = static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['hidden']; + $where[] = array( "{$col}!=-2 AND {$col} !=-3" ); + } + else if ( isset( static::$databaseColumnMap['approved'] ) ) + { + $col = static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['approved']; + $where[] = array( "{$col}!=-2 AND {$col}!=-3" ); + } + } /* Future items? */ if ( \in_array( 'IPS\Content\FuturePublishing', class_implements( \get_called_class() ) ) ) @@ -3321,18 +3338,28 @@ abstract class _Item extends \IPS\Content $categories = array(); $lookupKey = md5( $containerClass::$permApp . $containerClass::$permType . $permissionKey . json_encode( $member->groups ) ); + /* EME: Switch to use exclusion instead of inclusion */ if( !isset( static::$permissionSelect[ $lookupKey ] ) ) { static::$permissionSelect[ $lookupKey ] = array(); - $permQuery = \IPS\Db::i()->select( 'perm_type_id', 'core_permission_index', array( "core_permission_index.app='" . $containerClass::$permApp . "' AND core_permission_index.perm_type='" . $containerClass::$permType . "' AND (" . \IPS\Db::i()->findInSet( 'perm_' . $containerClass::$permissionMap[ $permissionKey ], $member->permissionArray() ) . ' OR ' . 'perm_' . $containerClass::$permissionMap[ $permissionKey ] . "='*' )" ) ); + + $permQueryJoinContainer = (bool) ( \count( $containerWhere ) ); + $clubWhere = ""; /* If we cannot access clubs, skip them */ if ( \IPS\IPS::classUsesTrait( $containerClass, 'IPS\Content\ClubContainer' ) AND !$member->canAccessModule( \IPS\Application\Module::get( 'core', 'clubs', 'front' ) ) ) { - $containerWhere[] = array( $containerClass::$databaseTable . '.' . $containerClass::$databasePrefix . $containerClass::clubIdColumn() . ' IS NULL' ); + $permQueryWhere = array( "core_permission_index.app='" . $containerClass::$permApp . "' AND core_permission_index.perm_type='" . $containerClass::$permType . "' AND (" . $containerClass::$databaseTable . '.' . $containerClass::$databasePrefix . $containerClass::clubIdColumn() . " IS NOT NULL OR !(" . \IPS\Db::i()->findInSet( 'perm_' . $containerClass::$permissionMap[ $permissionKey ], $member->permissionArray() ) . ' OR ' . 'perm_' . $containerClass::$permissionMap[ $permissionKey ] . "='*' ))" ); + $permQueryJoinContainer = true; } + else + { + $permQueryWhere = array( "core_permission_index.app='" . $containerClass::$permApp . "' AND core_permission_index.perm_type='" . $containerClass::$permType . "' AND !(" . \IPS\Db::i()->findInSet( 'perm_' . $containerClass::$permissionMap[ $permissionKey ], $member->permissionArray() ) . ' OR ' . 'perm_' . $containerClass::$permissionMap[ $permissionKey ] . "='*' )" ); + } + + $permQuery = \IPS\Db::i()->select( 'perm_type_id', 'core_permission_index', $permQueryWhere ); - if ( \count( $containerWhere ) ) + if( $permQueryJoinContainer ) { $permQuery->join( $containerClass::$databaseTable, array_merge( $containerWhere, array( 'core_permission_index.perm_type_id=' . $containerClass::$databaseTable . '.' . $containerClass::$databasePrefix . $containerClass::$databaseColumnId ) ), 'STRAIGHT_JOIN' ); } @@ -3347,11 +3374,7 @@ abstract class _Item extends \IPS\Content if( \count( $categories ) ) { - $where[] = array( static::$databaseTable . "." . static::$databasePrefix . static::$databaseColumnMap['container'] . ' IN(' . implode( ',', $categories ) . ')' ); - } - else - { - $where[] = array( static::$databaseTable . "." . static::$databasePrefix . static::$databaseColumnMap['container'] . '=0' ); + $where[] = array( static::$databaseTable . "." . static::$databasePrefix . static::$databaseColumnMap['container'] . ' NOT IN (' . implode( ',', $categories ) . ')' ); } } @@ -3363,8 +3386,10 @@ abstract class _Item extends \IPS\Content $select = \IPS\Db::i()->select( 'COUNT(*) as cnt', static::$databaseTable, $where, NULL, NULL, $groupBy, NULL, $queryFlags ); if ( $joinContainer AND isset( static::$containerNodeClass ) ) { + /* EME: Removed the $containerWhere from the join because it is now in the where clause. + We are now using $containerWhere to exclude forums that shouldn't be visible. */ $containerClass = static::$containerNodeClass; - $select->join( $containerClass::$databaseTable, array_merge( $containerWhere, array( static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['container'] . '=' . $containerClass::$databaseTable . '.' . $containerClass::$databasePrefix . $containerClass::$databaseColumnId ) ) ); + $select->join( $containerClass::$databaseTable, array( static::$databaseTable . '.' . static::$databasePrefix . static::$databaseColumnMap['container'] . '=' . $containerClass::$databaseTable . '.' . $containerClass::$databasePrefix . $containerClass::$databaseColumnId ) ); } if ( $joinComments ) { Could someone at Invision answer whether it would cause any issues to temporarily revert this method to the previous that worked for us? Or is there a lot of changes in that method dependent on other changes done elsewhere in the same or later version, so I would have to hunt down a lot of other stuff?

I upgraded from version 4.7.13 to 4.7.16 on 19th April, and we have since been having trouble with both reduced performance and more downtime. Our server host has looked into this and reported back a major increase in slow queries, major increase in CPU usage for the database server overall and less usage of MySQL Sorts. It seems this increase can be tracked down to methods using IPS\Content\_Item::getItemsWithPermission:317. We have a custom controller and a frontpage widget utilizing this method. Anyone else experienced this, or have any tips on what could be going on? Hope Invision could be interested in lending a hand investigating this too Best regards, Preben

I assume any tables installed by the plugin will remain in place? And you are just speaking about the table listing the plugins?