Ibragim Pupkevich Posted December 28, 2016 Posted December 28, 2016 Dear IPS team, I'm experiencing quite nasty troubles with my IPS 4.1 self-hosted community, which is quite heavy in terms of DB size (>4GB already) and daily page views (>70k). Even for 8-core Intel Core i7 CPU + 32GB RAM + SSD it appears to be quite hard job to run IPS engine scripts, server goes down quite often running php-fpm environment. Do you know any examples of other high-loaded communities running IPS 4.1 suite? Maybe their administrators can share some experience? OPcache helps a lot, but still... looks like IPS suite engine is very heavy. Different from older 3.x versions We have faced even higher page views rate 2 years ago, when we used IPB 3.4. But we've never faced any performance troubles like we face now. And it's not DDoS or any attack-related behaviour, already checked for that. Thx.
tolik777 Posted December 29, 2016 Posted December 29, 2016 Hi! I have DB Size over 8Gb, daily page views over 100k. Online users 1000-1500 on peaks. And 2 000 000 posts, 100k registered users. My forum is: http://forum.cxem.net/ 4.1.17 very hard VS 3.4... Now, my server is: i7-6700, 64 GB DDR4, 2 x 250GB SSD... The third version worked fine for less configuration. This version 4.1.x is slower than running on more powerful hardware.... If I off cache (I use Redis) - SQL immediately falls... I use NGINX + PHP 7.0 FPM. I do not recommend Apache on high load. And What do you use apache or nginx? php 5 or php 7?
mark007 Posted December 29, 2016 Posted December 29, 2016 I had massive performance problems with IPB 4.1.17.1. Then I disabled memcached, installed ZendOPCache and PHP 7. Data Storage method: MySQL Caching method: no Cache page output for guests: no -------- MySQL Query Cache 96MB max_heap_table_size=36Mtable_open_cache=2048 query_cache_limit = 2 M -> now IPB 4 is running faster then 3.4.8 !!
Ibragim Pupkevich Posted December 29, 2016 Author Posted December 29, 2016 It seems same sh*t for everyone Thanks for responses, guys!
Management Lindy Posted January 9, 2017 Management Posted January 9, 2017 Those having problems with Redis or Memcache, could you please submit a ticket? We are undergoing a comprehensive performance audit and would like to review your configuration. Thank you.
Colonel_mortis Posted January 9, 2017 Posted January 9, 2017 I use redis along with some custom plugins to move more things into redis (which reduced response times by >30% from what I remember) on a high traffic (10k peak online users, normally closer to 1-2k online at a time), and the mean server response time is ~100ms (median is <50ms). One thing that made a massive difference was removing the Who's Online block. Sure, it makes the site look good because lots of people are online, but it had a very significant impact on performance.
Janyour Posted January 12, 2017 Posted January 12, 2017 We have an optimized database of 700mb we are using serverpilot offering us a very optimized server with apache/Nginx !
SecondSight Posted January 12, 2017 Posted January 12, 2017 Hello ! I have about 800 users online and I don't use any cache software (like xCache). My server has got 2 SSD disks, 64 Gb RAM and the processor is Intel(R) Xeon(R) CPU E5-1650 v3 @ 3.50GHz. My database is now 11 Gb big... With IPS Community 3.4, everything was working fine because I had a table in Memory and two other tables were InnoDB (these were the tables causing problems - If I remember well, it was sessions and core_item_markers tables, not sure). Regarding IPS Community4, everything worked fine after I changed all tables from MyISAM to InnoDB. I also noticed each time I upgraded to a new IPS Community version (for instance, from IPS Community 3.4 to version 4), I had Apache performance problems for some time. I suspect search engine bots to be responsible for this (anyway in the logs I found there were too many connections for bots).
sudo Posted January 12, 2017 Posted January 12, 2017 12 hours ago, Janyour said: We have an optimized database of 700mb we are using serverpilot offering us a very optimized server with apache/Nginx ! LOL slipping in your referral code with a google redirect link.
Janyour Posted January 12, 2017 Posted January 12, 2017 Just now, ZeroHour said: LOL slipping in your referral code with a google redirect link.
Ibragim Pupkevich Posted January 26, 2017 Author Posted January 26, 2017 Guys, Please look at this: And share information about your number of forums (if you are also facing similar performance issues). Thanks.
bgk Posted February 16, 2017 Posted February 16, 2017 Hi All Any update here? I recently upgraded to 4.1.18.2 and am seeing terrible performance ...
ExiledVip3r Posted February 16, 2017 Posted February 16, 2017 I have a community of currently ~1,500,000 members, ~1,000,000 daily pageviews, and a 9gb database for the forums. I really want to upgrade, and I've tried twice now to upgrade my community; first to a midlife 4.0, and again to a midlife 4.1. They both ran like complete and utter crap. When not crashing the servers, they were ideally pushing 25% of the load times I got using 3.4. Even after optimizing them to the best of my knowledge. Using the same server environment that runs perfectly smooth with 3.4 with power to spare. It has been a complete and utter farce with 4.0+ to the point that I've been forced to apologize and revert the community back to the pre-"upgrade" backups both times after trying 4.0 and 4.1 for a full month of sheer stress. I've had to reverse engineer crap design and code both in hopes of making them work as expected. I had hoped 4.2 would finally get there with performance, but now I'm being shamed in the 3.4 admin panel that I'll stop getting security updates for the version that actually consistently works. Nothing irritates me more than using out of date in software and/or security... So I'm starting to look at alternatives at this point as I get flashbacks to my horrible experience with vB3... Or more likely just start replacing it all with something of my own...
RevengeFNF Posted February 16, 2017 Posted February 16, 2017 8 minutes ago, ExiledVip3r said: I have a community of currently ~1,500,000 members, ~1,000,000 daily pageviews, and a 9gb database for the forums. I really want to upgrade, and I've tried twice now to upgrade my community; first to a midlife 4.0, and again to a midlife 4.1. They both ran like complete and utter crap. When not crashing the servers, they were ideally pushing 25% of the load times I got using 3.4. Even after optimizing them to the best of my knowledge. Using the same server environment that runs perfectly smooth with 3.4 with power to spare. It has been a complete and utter faeces show with 4.0+ to the point that I've been forced to apologize and revert the community back to the pre-"upgrade" backups both times after trying 4.0 and 4.1 for a full month of sheer stress. I've had to reverse engineer faecesty design and code both in hopes of making them work as expected. I had hoped 4.2 would finally get there with performance, but now I'm being shamed in the 3.4 admin panel that I'll stop getting security updates for the version that actually consistently works. I'm starting to look at alternatives at this point as I get flashbacks to my horrible experience with vB3... Can you give us details of the optimization you were using? For example, your tables were in InnoDB? Nginx+php-fpm? etc etc
ExiledVip3r Posted February 16, 2017 Posted February 16, 2017 35 minutes ago, RevengeFNF said: Can you give us details of the optimization you were using? For example, your tables were in InnoDB? Nginx+php-fpm? etc etc Debian 8, 16gb+8cores+ssd, Nginx mainline, MariaDB 10.1 InnoDB, PHP-FPM 5.6/7.0 (tried both, but 7.0 was seemingly only loosely supported at the time), opcache (which was screwed up initially...), and tried pretty much every caching combination possible in the admin panel from nothing, redis, memcached, sql, and file. EDIT: Currently running with a dedicated remote SQL server, which wasn't the case during my attempted 4.0-4.1 upgrades. I doubt this would be the deciding factor now though, based off of my previous observations during upgrade, though even it were the ultimate solution, it's a poor one imo.
inkredible Posted February 16, 2017 Posted February 16, 2017 On 9.1.2017 at 4:54 AM, Lindy said: Those having problems with Redis or Memcache, could you please submit a ticket? We are undergoing a comprehensive performance audit and would like to review your configuration. Thank you. I would be happy to see an announcement describing the improvements resulted by this performance audit. It's great to hear such experiences from other fairly large communities. I am running a community with 150k members only, but we do have very large peaks with up to 2k live users (according to analytics) hence I rely on experiences and feedbacks from others in order to optimize the performance of my community as well.
ProSkill Posted February 19, 2017 Posted February 19, 2017 He's my experience for what it's worth. My site gets 15K - 20K visitors per day and I have been having serious slowness issues. Initially my site was running on a VPS with 8GB of RAM and 24 shared cores. I ran into a storage issue where the VPS only allowed up to a 200GB of storage so I had to move to a different server. I first moved to a dedicated server with 4 cores and 8 GB of RAM. I thought being on a VPS would provide better performance, but boy was I wrong. The site quickly overwhelmed the 4 cores and was basically unusable. I then decided my only option was to move back to the VPS, and offload attachments to Amazon S3. Well, Amazon S3 is insanely expensive for bandwidth it was running about 400 per month just for S3 storage, and the site was still slow. So my next try was to move the database over to Amazon RDS which again was insanely expensive it was $11 for about 8 hours of use (imagine the monthly cost) and was very slow due to latency. I then decide to spin up another VPS just for the database, and move the site and attachments back to a dedicated server. Well everything was still very slow and PHP was eating up all the CPU even with no MySQL running on the server, I was getting 64 Server load with 4 cores! So here is the fix and it's like night and day. I installed Easy Apache 4 and installed php 7 with memcached. The site is now faster than it's ever been even with 2k users on. So if your site is slow I would highly recommend upgrading to PHP 7. This took me about 2 months to figure out but, the site is blazing fast now. My current setup is: 4 GB 24 core VPS that only runs MySQL, and a 4 core 8 GB dedicated server that runs the forum.
marklcfc Posted February 19, 2017 Posted February 19, 2017 How did you install memcached with php 7 as I'm always told it is not compatible?
RevengeFNF Posted February 19, 2017 Posted February 19, 2017 6 hours ago, ProSkill said: He's my experience for what it's worth. My site gets 15K - 20K visitors per day and I have been having serious slowness issues. Initially my site was running on a VPS with 8GB of RAM and 24 shared cores. I ran into a storage issue where the VPS only allowed up to a 200GB of storage so I had to move to a different server. I first moved to a dedicated server with 4 cores and 8 GB of RAM. I thought being on a VPS would provide better performance, but boy was I wrong. The site quickly overwhelmed the 4 cores and was basically unusable. I then decided my only option was to move back to the VPS, and offload attachments to Amazon S3. Well, Amazon S3 is insanely expensive for bandwidth it was running about 400 per month just for S3 storage, and the site was still slow. So my next try was to move the database over to Amazon RDS which again was insanely expensive it was $11 for about 8 hours of use (imagine the monthly cost) and was very slow due to latency. I then decide to spin up another VPS just for the database, and move the site and attachments back to a dedicated server. Well everything was still very slow and PHP was eating up all the CPU even with no MySQL running on the server, I was getting 64 Server load with 4 cores! So here is the fix and it's like night and day. I installed Easy Apache 4 and installed php 7 with memcached. The site is now faster than it's ever been even with 2k users on. So if your site is slow I would highly recommend upgrading to PHP 7. This took me about 2 months to figure out but, the site is blazing fast now. My current setup is: 4 GB 24 core VPS that only runs MySQL, and a 4 core 8 GB dedicated server that runs the forum. You said you were paying 400$ just for the S3. With that kind of money you can buy a Dedicate Server with a lot more than just 4 cores. For example, for 240$ you can buy a Dedi Server with 20 Cores/40 Threads, 256 GB of Ram and 2x 480Gb SSD. 1 hour ago, marklcfc said: How did you install memcached with php 7 as I'm always told it is not compatible? If you use Centos, this one for Remi repo works perfectly: Name : php-pecl-memcached Arch : x86_64 Version : 3.0.2 Release : 1.el7.remi.7.1 Size : 267 k Repo : installed From repo : remi-php71 Summary : Extension to work with the Memcached caching daemon URL : http://pecl.php.net/package/memcached License : PHP Description : This extension uses libmemcached library to provide API for communicating : with memcached servers. : : memcached is a high-performance, distributed memory object caching system, : generic in nature, but intended for use in speeding up dynamic web : applications by alleviating database load. : : It also provides a session handler (memcached). : : Package built for PHP 7.1.
Qubabos Posted February 19, 2017 Posted February 19, 2017 On 12.01.2017 at 8:11 PM, SecondSight said: I also noticed each time I upgraded to a new IPS Community version (for instance, from IPS Community 3.4 to version 4), I had Apache performance problems for some time. I suspect search engine bots to be responsible for this (anyway in the logs I found there were too many connections for bots). I'm also facing the same problem - 2 different communities, 2 different servers from 2 diferent providers and I get http 429 too many request for googlebot on urls containing string *tags* (hxxp://siteurl/tags/*) - reported 2 times already to IPS and their always saying that is server issue (both run on dualcore xeon with 4 GB of ram, ssd disks, centos and debian and are not heavy loaded with traffic)
ProSkill Posted February 19, 2017 Posted February 19, 2017 7 hours ago, RevengeFNF said: You said you were paying 400$ just for the S3. With that kind of money you can buy a Dedicate Server with a lot more than just 4 cores. For example, for 240$ you can buy a Dedi Server with 20 Cores/40 Threads, 256 GB of Ram and 2x 480Gb SSD. If you use Centos, this one for Remi repo works perfectly: Name : php-pecl-memcached Arch : x86_64 Version : 3.0.2 Release : 1.el7.remi.7.1 Size : 267 k Repo : installed From repo : remi-php71 Summary : Extension to work with the Memcached caching daemon URL : http://pecl.php.net/package/memcached License : PHP Description : This extension uses libmemcached library to provide API for communicating : with memcached servers. : : memcached is a high-performance, distributed memory object caching system, : generic in nature, but intended for use in speeding up dynamic web : applications by alleviating database load. : : It also provides a session handler (memcached). : : Package built for PHP 7.1. I run an adult site and I don't trust many of the dedicated server hosting companies. A lot of them just don't seem to be run very professionally and they aren't afraid to suspend accounts over little things (one of my accounts got suspended because of a "DDoS attack" when really it was just a lot of legitimate traffic). Also, they often times don't have the same backup options, security features, or support. So yes a dedicated server with great specs might be cheaper but it's also more risky imho.
waccoe.com Posted February 21, 2017 Posted February 21, 2017 @ASTRAPI helped us with our community. over 20k registered members between 1500-2,500 active users per day
ASTRAPI Posted February 22, 2017 Posted February 22, 2017 I manage also forums with a lot more traffic....At pick times we have 4.000 users online at the same time :-) More than 20.000.000 posts and database size at 80GB and users produce almost 2 GB of data daily. Most reports for IPB 4 bad performance are related to use of many addons/apps. Also it will be an issue if an addon/app is bad coded without performance in mind... Also other parameters for bad performance is if users don't use any caching like zend opcache or Memcached.... One more reason is that even users use very good software on the server they do not optimize it! There is a ton of related adjustments as for very high traffic forums users may get limited by the system OS like tcp limitations, open file limitations and so on...
cocoliso Posted February 23, 2017 Posted February 23, 2017 My forum has more than 24.000 users, 12-15.000 page views per day. Wich is better, Apache o Nginx?
RevengeFNF Posted February 23, 2017 Posted February 23, 2017 19 minutes ago, cocoliso said: My forum has more than 24.000 users, 12-15.000 page views per day. Wich is better, Apache o Nginx? It depends on your need, but generals nginx + php-fpm is faster and lighter than apache with mod_php.
Recommended Posts
Archived
This topic is now archived and is closed to further replies.