tforums

Sorry. I have not kept up with these changes because everything has been running so well until recently. You are right, a CDN is a better option for all the long term unknowns I am worried about.

Thanks. Like you said, the ideal solution is to maintain the HEAD option. IPB already caches a lot of things, so it should be possible to at least cache the header for the /discovery/ pages that are being hit constantly to check for updates. This is only going to get worse as AI crawlers are deployed. Forums communities are being targeted for their embedded knowledge. Even if your site isn't getting hit a lot, I am sure it is adding a some load to the server. The IPB cloud service is likely burning a lot of CPU because of this. Do us all a favor please! 😄

Yes, I said that. Which is why I am trying to see if IPB can cache HEAD requests instead of hitting the DB every time.

FWIW, I don't think this is an attack so much as we are a very old, large, and popular community. Yes see above. I could send you logs all day. Probably not something I should post here. LMK where to send them if you want to take a look.

... I cannot add to the post for some reason. But I am seeing some very abusive activity from Facebook and other scrapers where they slam a site with 10 requests from 10 different IPs. This appears to be a shortcut for them doing replication. Its much faster and easier to hit us 10 times than read once, and then sync their backend.

In the age of AI and Social Media, there are more bots than ever. I recently noticed we are giving up 70% of our server load to HEAD requests. This appears to be fueling a constant load on the database even when there is very little activity from members. If you are seeing a kind of pumping activity of load rise and fall, it is probably from this. I have blocked HEAD requests to mitigate this activity because it is being done by dynamic IP sources without user agents and or ignoring robots.txt. However, this is not ideal as HEAD should be saving load from GET requests. Seems to me the IPB system could somehow mitigate this or is everything, even the headers, coming from the system DB? IMO, IPB really needs to focus some tools on how to deal with bots and scraping. The AI boom is literally inciting others to machine-learn our community activity for who knows what uses. This is not only hurting our sever performance, it is abusing the community and members in ways we cannot forsee.

IDK what happened, but it is working now. *knock on wood*

I see the problem. The applications/cms files are not included in the current version. I don't see how / where to download those separately. Did something change?

Thanks. Just tired that. Still getting the same error.

Thanks, I went ahead and activated display errors in the conf, and see this showing up:

I am the admin and we are. The Locked Task: clubrebuild error is not related?

Thanks, but there are no errors showing up for the clubs in the Admin or Httpd server logs.

We are trying to remove the Classifieds 2.1.1 Application that is disabled since the Marketplace shut down. Unfortunately, the Uninstall option goes to a blank page and nothing is removed. Is there a process to remove Apps manually?

I am seeing this Alert. Everything else is running fine. Running manually tells me it is already running already and cannot be run again until the lock is removed.

The separate Clubs pages are all blank now since the last update. Caches have been cleared and no errors showing in admin console. Is there a fix for this?

Just a couple of side block with HTML. Yes, it is pulling the latest topics as a feed.

I can confirm this issue on two other installs. The SQL query needs to be optimized, please, so we can restore Portal to our homepages. Thanks!

Thanks, I found the support forum and posted there.

Having CPU load issues when Portal is made the default homepage after latest IPB upgrade. Please see the topic here. https://invisioncommunity.com/forums/topic/471698-database-process-running-constantly-overloading-cpu/#comment-2923362 I have tried re-installing the Portal 1.8.1, but the problem is still there.

I am trying to, but any PM to the developer @Michael.J shows a forums error:

UPDATE - The process list hanging / CPU load issue seems to be tied to the Portal app being set as the forum homepage. When I set the home page to the IPB System, there are no "Sending data / Creating sort index" tasks and CPU load goes back to normal. I have confirmed the load issue can be reproduced by switching Portal back to the default forum homepage. I have disabled the Portal for Guests, and that seems to have cooled things off, but my guess is there's still something not right. What's odd is we've had this app installed at the homepage for years without issues until the last update.

SORRY! I spoke too soon. Seeing the hanging processes again in the DB after some time. Disabling Portal App stops them, so I am back on trying to debug this install.

UPDATE - It must have been a cached configuration issue. After updating the settings, it appears to be working without adding any load.

Thanks! It turns out to be the Portal 1.8.1 app causing the problem. This is odd because it is marked as 4.7.x compatible and works fine in our other installs.

Is there a Bug in the system? I see a lot of these errors in the system logs: