NexusMods

Hi there, We unfortunately have come across some abuse of the private message / conversation feature. What happens: Users can message each other, then block the others / themselves from a conversation. As the conversation then has no participants, it's deleted and we have no trace, making it difficult to moderate. Expected behaviour: When there are only two participants, users shouldn't be able to remove the other member (to cause the deletion), or there needs to be some way of verifying that the PM did exist as a forum admin. How to reproduce: Here is the steps to reproduce from our team: Log into AccountA - send a PM to AccountB Within the PM UI, select AccountB from the list of participants and select "Remove from conversation". Delete the message from your inbox as AccountA. Log into AccountB - see that this PM "never existed" but you still have an email confirming it was sent to you. The class / method in question is Messenger\Conversation::deauthorize If you need any more information, please let us know.

Hi, We've been having issues every 6 hours when the badge status cache expires and they regenerate. This causes our (rather large) DB to hit 100% CPU and the forums become unresponsive. Looking into this, the issue is that multiple threads will try to get the badge stats at once when they expire: \IPS\core\Achievements\_Badge::getBadgeStats() We have approximately 40 concurrent requests trying to regenerate these stats every 6 hours. As a work around, we are temporarily updating the stats expiration while we re-generate the statistics, this results in it only running once per 6 hours (instead of 40 times) if( !isset( $stats['expiration'] ) OR time() > $stats['expiration'] OR !isset( $stats['badgeCount'][ $this->id ] ) ) { // Write the old stats back to the store while we regenerate them $stats['expiration'] = time() + 60; \IPS\Data\Store::i()->badgeStats = json_encode( $stats ); $stats = NULL; }

Hi, okay, that's interesting - mine still returns UNKNOWN_ERROR with the exact same values you are sending: If there is something relating to warning actions that matches up with where I think its erroring trying to send form a specific output. Please keep me in the loop, thanks. /* We do this after the action is checked because the moderator may not be able to override the action, in which case the actual values won't have been submitted */ foreach( $values['warn_punishment'] AS $p ) { if ( $values['warn_' . $p ] === NULL ) { \IPS\Output::i()->error( 'no_warning_action_time', '1C150/2', 403, '' ); } }

Hi, please could I get a resolution on this. I believe its attempting to output this message. I'm not sure what this is relating to though. You must specify a timeframe for the actions applied, or select "Indefinitely"

Hi, you are sending a suspendPosts query param, not restrictPosts as your documentation states? https://invisioncommunity.com/developers/rest-api?endpoint=core/members/POSTitem_warnings

Thanks for the swift reply and pointing me in the right direction. I do actually see two System errors produced:

To show its the same result with -1:

Hi, Hope you are well. I'm experiencing an issue when trying to send any datetime values in query params when creating a member warning with a POST request to the core/members/{member_id}/warnings. According to the documentation I'm trying to form a url including suspend and/or restrict values which can be datetime|-1|null. However, when sending a datetime (or -1) I receive an UNKNOWN_ERROR response from the invision API. I've investigated this as much as I can and believe its being produced in Warning.php within a foreach loop where its trying to output 'no_warning_action_time' error - although, only the UNKNOWN_ERROR response is actually received. I may be incorrect on this though so seeking advice on next steps. I suspect its something to do with the datetime format I'm sending, any help would be appreciated. The formed URL and response looks as below (in postman):

Hi, we are seeing issues with the viewupdates task in the system logs: ArgumentCountError: Wrong parameter count for Redis::zRem() (0) #0 /var/www/html/system/Redis/Redis.php(250): Redis->zRem('topic_views') #1 /var/www/html/applications/core/tasks/viewupdates.php(86): IPS\_Redis->__call('zRem', Array) #2 /var/www/html/system/Task/Task.php(375): IPS\core\tasks\_viewupdates->IPS\core\tasks\{closure}() #3 /var/www/html/applications/core/tasks/viewupdates.php(38): IPS\_Task->runUntilTimeout(Object(Closure)) #4 /var/www/html/system/Task/Task.php(274): IPS\core\tasks\_viewupdates->execute() #5 /var/www/html/system/Task/Task.php(237): IPS\_Task->run() #6 /var/www/html/applications/core/interface/task/task.php(72): IPS\_Task->runAndLog() #7 {main} It seems adding an !empty check on the $redis array in viewupdates.php fixes the error: if( \is_array( $redis ) && !empty($redis) ) { foreach ( $redis as $data => $count )

Hi Nathan, Thanks for your reply. That post seems mainly concerned with OAuth and how best to integrate with Invision. It does though confirm that making changes to the database schema is a bad idea. We are concerned with reliability and performance. Our understanding is that for replication to be reliable and performant it is recommended to have primary keys (although unique indexes will work as well https://blog.jcole.us/2013/05/02/how-does-innodb-behave-without-a-primary-key/). Is there a technical reason why not to add primary keys to the above tables? Thanks,

Just to add a little more background. Digital Ocean issue this warning: It also requires the setting `sql_require_primary_key` needs to be disabled in order to setup the DB. By default this is off in normal MySQL installs, but with Digital Ocean it defaults to on due to the data loss risk and requires this API call https://docs.digitalocean.com/reference/api/api-reference/#operation/databases_patch_config.

Hi, We were wondering if there is a reason that the following tables do not have primary keys? ibf_core_acp_tab_order - unique (id) ibf_core_alerts_seen - unique (seen_alert_id, seen_member_id) ibf_core_automatic_moderation_pending - unique (pending_object_class, pending_object_id) ibf_core_cache - unique (cache_key) ibf_core_follow_count_cache - unique (id, class) ibf_core_item_markers - unique (item_key, item_member_id, item_app) ibf_core_item_member_map - unique (map_class, map_item_id, map_member_id) ibf_core_members_logins - unique (member_id, member_date) ibf_core_oauth_server_access_tokens - unique (client_id, access_token) ibf_core_oauth_server_authorization_codes - unique (client_id, code) ibf_core_reputation_leaderboard_history - unique (leader_date, leader_member_id) ibf_core_search_index_item_map - unique (index_item_id, index_class, index_author_id) ibf_core_security_answers - unique (answer_question_id, answer_member_id) ibf_core_tags_cache - unique (tag_cache_key) ibf_core_tags_perms - unique (tag_perm_aai_lookup) ibf_forums_view_method - unique (member_id, type) We believe that there would be a benefit to having primary keys when using MySQL in a cluster to replicate changes. However, we don't want to manually add these and customise our setup making future upgrades riskier / harder. Thanks

Hi, We have an issue with the delete secondary group rest API endpoint The code does not appear to check if the found group key is false before calling unset, which results in the first secondary group being removed (As PHP is casting false to 0) We are currently working around this by checking if the $key is false before calling unset. $secondaryGroups = array_filter( explode( ',', $member->mgroup_others ) ); $key = array_search( $group->g_id, $secondaryGroups ); if (false !== $key) { unset($secondaryGroups[$key]); } e.g: member 1's secondary groups are "5,6,7" call DELETE /core/members/1/secgroup/2 member 1's groups are now 6,7 and should still be "5,6,7"

Thanks!

Hi Marc, That's right it would be load balanced. Do you know if there is any other state that we need to consider? We know of: Sessions (logins) seem to be handled by Redis or the Database uploads would require shared storage datastore would require shared storage Thanks for all of your help, Theo

Hi Marc, Thanks for the rapid replies. We're self hosting the software ourselves but we'd like to confirm that the license terms for Invision support the use-case of horizontally scaling We've looked at the code and it seems like session state is handled in either Redis or by the Database, so it looks like technically we should be able to horizontally scale. However, we're not sure if there is any other state we would need to be aware of. Thanks

Hi Marc, This is one installation, a single URL. However, it would be horizontally scaled to handle load if required. Do your license terms allow for this? Yes. We're aware of the uploads and datastore directories that would need to be shared. We're not sure how the session (login) state is handled. Is there any other state that would be required. Thanks

Hi, We would like to be able to horizontally scale Invision. Is it possible to run Invision with multiple instances? What state would need to be shared between instances to make this possible? Thanks

Thanks for the quick confirmation Marc!

When we say slow we're talking the difference between 10ms and 1ms, but that makes a significant difference to the overall time when there are millions of records to process during the v3 to v4 upgrade (specifically inserting each member during IPS\core\setup\upg_40000\_Upgrade::step1). Multibyte support is a good point, although it wasn't a problem during our upgrade as the prepared query wasn't using any multibyte chars.

We'd been looking into the possibility of assigning users into groups based on information from the oauth provider. It doesn't appear that there is any mechanism for that. Can anyone confirm if that's definitely the case, or there is some other technique to achieve that? We'd considered the mechanism to promote members into groups, but it doesn't appear to have a way of checking values set by a login handler. Thanks for any advice you can offer.

Sharing this in case it helps others, or can be merged upstream. While migrating a large forum we were encountering some poor performance that was narrowed down to DB.preparedQuery. Changing the substitution approach yielded a 10x speedup. Current implementation does: /* For NULL values, you can't bind, so we adjust the query to actually pass a NULL value */ $pos = 0; for ( $j=0; $j<$i; $j++ ) { $pos = mb_strpos( $query, '?', $pos ) + 1; } $query = mb_substr( $query, 0, $pos - 1 ) . 'NULL' . mb_substr( $query, $pos ); We replaced this with: // Replace the i'th '?' character with 'NULL'. $found = preg_match_all('/'.preg_quote('?').'/', $query, $matches, PREG_OFFSET_CAPTURE); if (false !== $found && $found >= $i) { $query = substr_replace($query, 'NULL', $matches[0][$i - 1][1], 1); } If there is a better place to raise this for upstream then please let me know.

Yep that works, getting results now that the schema is correct. I can only suggest to others performing a v4 migration that they only configure ES after completing all the background rebuild tasks. Thanks for the quick response!

This might be a problem of having configured ES while rebuilding tasks were still going (we're upgrading from v3 to v4). The ES schema includes mapping: "index_class": { "type": "text", "fields": { "keyword": { "ignore_above": 256, "type": "keyword" } } } However if I rebuild the search index on a smaller database the mapping contains: "index_class": { "type": "keyword" } Will delete and rebuild the search index since completing the migration background tasks.

Yes, the query is made to ES and there are no hits. e.g. response is {"took":3,"timed_out":false,"_shards":{"total":1,"successful":1,"skipped":0,"failed":0},"hits":{"total":{"value":0,"relation":"eq"},"max_score":null,"hits":[]}}