Invision Community 4: SEO, prepare for v5 and dormant account notifications By Matt Monday at 02:04 PM
SG Staff Posted November 8, 2012 Posted November 8, 2012 I upgraded and now things are running as slow as slow can be. I did the security patch also. I checked my server (VPS) and my CPU keeps spiking all the way up, close to 100%. I have tried restarting Apache & MySQL and rebooting the whole server. Neither helped. I originally posted in the IP.Board support forum, but it was suggested I post here for help. Thanks :)
Management Charles Posted November 8, 2012 Management Posted November 8, 2012 You may want to try this Knowledge Base article: http://community.invisionpower.com/resources/guides.html/_/knowledge-base/large-topics-causing-high-load-r234
AlexJ Posted November 9, 2012 Posted November 9, 2012 Post the output of top command if above fix doesn't help. Might be iowait or RAM issue?
SG Staff Posted November 9, 2012 Author Posted November 9, 2012 I tried the new topics.php file as suggested, but it didn't seem to make a difference. We don't have any topics quite that large, our largest are around 25-26k and we only have a few of those. My RAM is pretty free, it really seem to just be a CPU spike - it goes between 0% and 97+% almost instantly, and everywhere in between Maxing out - top - 05:32:40 up 11 min, 1 user, load average: 5.95, 5.63, 3.23 Tasks: 114 total, 11 running, 103 sleeping, 0 stopped, 0 zombie Cpu(s): 97.6%us, 1.7%sy, 0.0%ni, 0.0%id, 0.2%wa, 0.0%hi, 0.0%si, 0.5%st Mem: 2097328k total, 1293448k used, 803880k free, 34124k buffers Swap: 1052248k total, 0k used, 1052248k free, 817956k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4057 apache 15 0 28396 15m 1756 S 17.6 0.8 0:29.28 httpd 4182 apache 15 0 28532 15m 1764 S 17.3 0.8 0:28.10 httpd 4198 apache 16 0 28400 15m 1756 S 17.3 0.8 0:25.41 httpd 4193 apache 15 0 28396 15m 1756 S 17.0 0.8 0:24.29 httpd 4200 apache 15 0 28532 16m 1764 S 17.0 0.8 0:26.98 httpd 4054 apache 15 0 28396 15m 1756 S 16.7 0.8 0:21.03 httpd 4183 apache 15 0 28396 15m 1756 S 16.7 0.8 0:25.54 httpd 4199 apache 15 0 28400 15m 1756 S 16.4 0.8 0:28.51 httpd 4197 apache 18 0 28404 15m 1764 R 11.6 0.8 0:27.03 httpd 4058 apache 15 0 28396 15m 1756 S 11.0 0.8 0:31.48 httpd 4187 apache 15 0 28396 15m 1764 S 9.4 0.8 0:24.66 httpd 4059 apache 15 0 28396 15m 1756 S 9.1 0.8 0:27.98 httpd 4203 apache 15 0 28396 15m 1756 S 8.2 0.8 0:25.93 httpd 4056 apache 16 0 28400 15m 1764 R 5.3 0.8 0:32.14 httpd 4188 apache 18 0 28400 15m 1756 R 2.8 0.8 0:24.93 httpd 4829 seaso2 16 0 44888 29m 6044 S 1.9 1.4 0:00.52 php-cgi 4196 apache 19 0 28404 15m 1756 R 1.6 0.8 0:28.57 httpd 4820 seaso2 16 0 44888 29m 6108 R 1.3 1.4 0:00.49 php-cgi 4971 seaso2 18 0 10572 1460 1156 R 0.3 0.1 0:00.01 php-cgi 1 root 15 0 2084 648 552 S 0.0 0.0 0:00.01 init 2 root RT -5 0 0 0 S 0.0 0.0 0:00.01 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0 4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0 5 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/0 6 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khelper 7 root 13 -5 0 0 0 S 0.0 0.0 0:00.00 kthread 9 root 19 -5 0 0 0 S 0.0 0.0 0:00.00 xenwatch 10 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 xenbus 14 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/1 15 root 34 19 0 0 0 R 0.0 0.0 0:00.00 ksoftirqd/1 16 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/1 17 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/1 20 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/0 21 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/1 22 root 20 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/0 23 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/1 27 root 20 -5 0 0 0 S 0.0 0.0 0:00.00 khubd 29 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kseriod 93 root 15 0 0 0 0 S 0.0 0.0 0:00.00 pdflush 94 root 15 0 0 0 0 S 0.0 0.0 0:00.00 pdflush 95 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 kswapd0 96 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 aio/0 97 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 aio/1 238 root 20 -5 0 0 0 S 0.0 0.0 0:00.00 kpsmoused 266 root 19 -5 0 0 0 S 0.0 0.0 0:00.00 kstriped 279 root 10 -5 0 0 0 S 0.0 0.0 0:00.06 kjournald 301 root 11 -5 0 0 0 S 0.0 0.0 0:00.00 kauditd 329 root 19 -4 2168 596 380 S 0.0 0.0 0:00.06 udevd 655 root 14 -5 0 0 0 S 0.0 0.0 0:00.00 kmpathd/0 656 root 14 -5 0 0 0 S 0.0 0.0 0:00.00 kmpathd/1 657 root 14 -5 0 0 0 S 0.0 0.0 0:00.00 kmpath_handlerd 682 root 11 -5 0 0 0 S 0.0 0.0 0:00.00 kjournald 975 root 12 -4 12536 780 564 S 0.0 0.0 0:00.02 auditd 977 root 8 -8 12092 684 576 S 0.0 0.0 0:00.00 audispd 997 root 18 0 1736 576 488 S 0.0 0.0 0:00.00 syslogd 1000 root 15 0 1688 404 340 S 0.0 0.0 0:00.00 klogd 1010 root 15 0 2484 376 268 S 0.0 0.0 0:00.00 irqbalance 1042 named 16 0 50136 3692 2064 S 0.0 0.2 0:00.02 named 1097 root 19 0 4548 1204 1064 S 0.0 0.1 0:00.00 xe-daemon 1108 root 15 0 1672 424 356 S 0.0 0.0 0:00.00 courierlogger Min - top - 05:33:44 up 12 min, 1 user, load average: 3.62, 5.16, 3.24 Tasks: 111 total, 1 running, 110 sleeping, 0 stopped, 0 zombie Cpu(s): 2.2%us, 0.3%sy, 0.0%ni, 92.0%id, 5.3%wa, 0.0%hi, 0.2%si, 0.0%st Mem: 2097328k total, 1248596k used, 848732k free, 34240k buffers Swap: 1052248k total, 0k used, 1052248k free, 819252k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 1 root 15 0 2084 648 552 S 0.0 0.0 0:00.01 init 2 root RT -5 0 0 0 S 0.0 0.0 0:00.01 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/0 4 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/0 5 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/0 6 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khelper 7 root 13 -5 0 0 0 S 0.0 0.0 0:00.00 kthread 9 root 19 -5 0 0 0 S 0.0 0.0 0:00.00 xenwatch 10 root 13 -5 0 0 0 S 0.0 0.0 0:00.00 xenbus 14 root RT -5 0 0 0 S 0.0 0.0 0:00.00 migration/1 15 root 34 19 0 0 0 S 0.0 0.0 0:00.00 ksoftirqd/1 16 root RT -5 0 0 0 S 0.0 0.0 0:00.00 watchdog/1 17 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 events/1 20 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/0 21 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kblockd/1 22 root 20 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/0 23 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 cqueue/1 27 root 20 -5 0 0 0 S 0.0 0.0 0:00.00 khubd 29 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kseriod 93 root 15 0 0 0 0 S 0.0 0.0 0:00.00 pdflush 94 root 15 0 0 0 0 S 0.0 0.0 0:00.00 pdflush 95 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 kswapd0 96 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 aio/0 97 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 aio/1 238 root 20 -5 0 0 0 S 0.0 0.0 0:00.00 kpsmoused 266 root 19 -5 0 0 0 S 0.0 0.0 0:00.00 kstriped 279 root 10 -5 0 0 0 S 0.0 0.0 0:00.06 kjournald 301 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kauditd 329 root 19 -4 2168 596 380 S 0.0 0.0 0:00.06 udevd 655 root 14 -5 0 0 0 S 0.0 0.0 0:00.00 kmpathd/0 656 root 14 -5 0 0 0 S 0.0 0.0 0:00.00 kmpathd/1 657 root 14 -5 0 0 0 S 0.0 0.0 0:00.00 kmpath_handlerd 682 root 11 -5 0 0 0 S 0.0 0.0 0:00.00 kjournald 975 root 11 -4 12536 780 564 S 0.0 0.0 0:00.02 auditd 977 root 7 -8 12092 684 576 S 0.0 0.0 0:00.00 audispd 997 root 18 0 1736 576 488 S 0.0 0.0 0:00.00 syslogd 1000 root 15 0 1688 404 340 S 0.0 0.0 0:00.00 klogd 1010 root 15 0 2484 376 268 S 0.0 0.0 0:00.00 irqbalance 1042 named 16 0 50136 3692 2064 S 0.0 0.2 0:00.02 named 1097 root 18 0 4548 1204 1064 S 0.0 0.1 0:00.00 xe-daemon 1108 root 15 0 1672 424 356 S 0.0 0.0 0:00.00 courierlogger 1123 root 18 0 2004 640 520 S 0.0 0.0 0:00.00 authdaemond 1181 root 18 0 2004 252 132 S 0.0 0.0 0:00.00 authdaemond 1182 root 18 0 2004 252 132 S 0.0 0.0 0:00.00 authdaemond 1183 root 18 0 2004 252 132 S 0.0 0.0 0:00.00 authdaemond 1184 root 18 0 2004 252 132 S 0.0 0.0 0:00.00 authdaemond 1185 root 18 0 2004 252 132 S 0.0 0.0 0:00.00 authdaemond 1186 dbus 15 0 2768 856 680 S 0.0 0.0 0:00.00 dbus-daemon 1202 root 16 0 12748 1272 572 S 0.0 0.1 0:00.00 pcscd 1231 root 15 0 27264 1364 1056 S 0.0 0.1 0:00.00 automount 1250 root 15 0 7088 1080 672 S 0.0 0.1 0:00.00 sshd 1262 root 15 0 2740 860 680 S 0.0 0.0 0:00.00 xinetd 1294 root 23 0 4548 1244 1084 S 0.0 0.1 0:00.00 mysqld_safe
AlexJ Posted November 9, 2012 Posted November 9, 2012 Cpu(s): 97.6%us, 1.7%sy, 0.0%ni, 0.0%id, 0.2%wa, 0.0%hi, 0.0%si, 0.5%st 100% Hard drive usage? Run below commands and provide the results: 1. iostat 2. Run vmstat 3 for 10-15 seconds and post the results when u are experiencing high CPU usage. 3. mpstat -P ALL 4. dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync Do you have correct settings for your apache server? What are your settings for apache?
SG Staff Posted November 9, 2012 Author Posted November 9, 2012 1. iostat - command not found 2. vmstat 3 [root@vps ~]# vmstat 3 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------ r b swpd free buff cache si so bi bo in cs us sy id wa st 0 0 0 421280 62168 1170864 0 0 31 82 134 102 67 4 25 3 0 0 0 0 420352 62168 1170928 0 0 28 0 228 449 31 3 62 4 0 0 0 0 420352 62176 1170956 0 0 0 333 120 40 0 0 89 11 0 0 0 0 420432 62176 1170964 0 0 0 0 107 98 13 2 85 0 0 0 0 0 420432 62184 1170964 0 0 0 176 93 37 0 0 94 6 0 0 0 0 420228 62192 1170984 0 0 5 289 248 438 21 2 68 8 0 1 0 0 415256 62192 1170992 0 0 0 0 117 41 19 0 81 0 0 0 0 0 420220 62196 1171000 0 0 1 205 131 139 4 1 89 6 0 1 0 0 400612 62200 1171004 0 0 1 297 250 170 35 2 59 4 0 1 0 0 407928 62200 1171004 0 0 0 0 160 67 34 1 65 0 0 0 0 0 419972 62208 1171052 0 0 11 301 214 409 11 1 77 11 0 1 0 0 399140 62208 1171068 0 0 1 0 213 427 43 6 50 1 0 3. mpstat -P ALL - command not found (mpstat command, that is) 4. dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync My server just stares at me and blinks... :lol: No response at all, just seems to sit there. Maybe I need to wait a long time? Not sure how to check my apache settings, but that's not something I would have messed with (I am sure that's now obvious)
AlexJ Posted November 9, 2012 Posted November 9, 2012 If you have centos VPS: yum install sysstat If you have Debian apt-get install sysstat @4 - How long did you waited? Wait for 2-10 mins in worst case scenario. If you have to wait more then that if no process is running on your VPS then surely time to change your VPS host.
SG Staff Posted November 12, 2012 Author Posted November 12, 2012 I didn't even wait 2 mins. I am sure I gave up before that. I'll run this stuff and get back to you again. Thanks :)
SG Staff Posted November 12, 2012 Author Posted November 12, 2012 This is a quieter time of the day for us, but the CPU load is still pretty high. [root@vps ~]# iostat Linux 2.6.18-164.el5xen (vps.seasonedgamers.com) 11/12/2012 avg-cpu: %user %nice %system %iowait %steal %idle 39.58 0.05 2.32 5.55 0.20 52.31 Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn xvda 20.18 187.34 347.17 44418107 82311236 xvda1 0.00 0.01 0.00 1610 12 xvda2 0.00 0.00 0.00 1119 168 xvda3 20.18 187.33 347.17 44415090 82311056 [root@vps ~]# mpstat -P ALL Linux 2.6.18-164.el5xen (vps.seasonedgamers.com) 11/12/2012 05:06:53 AM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s 05:06:53 AM all 39.57 0.05 2.28 5.55 0.00 0.04 0.20 52.31 209.24 05:06:53 AM 0 46.97 0.05 2.65 8.40 0.00 0.03 0.23 41.65 146.42 05:06:53 AM 1 32.17 0.05 1.92 2.70 0.00 0.04 0.16 62.96 62.83 I simply didn't wait long enough... [root@vps ~]# dd if=/dev/zero of=test bs=64k count=16k conv=fdatasync 16384+0 records in 16384+0 records out 1073741824 bytes (1.1 GB) copied, 778.736 seconds, 1.4 MB/s
AlexJ Posted November 15, 2012 Posted November 15, 2012 I would definitely look for different VPS. Probably your VPS is overloaded and hence you are facing issue. 1.4MB/s is very poor I/O.
SG Staff Posted November 15, 2012 Author Posted November 15, 2012 Any suggestions? I've never loved Lunarpages.
Aussie Cable Posted November 15, 2012 Posted November 15, 2012 Any suggestions? I've never loved Lunarpages. I can only suggest IPS hosting.
handsoffsam Posted November 16, 2012 Posted November 16, 2012 Amazon EC2 is very good. But it's just a blank machine, no control panels or other admin wizard junk. You'll need to know how to operate a Linux/UNIX system. But as far as providing infrastructure as a service goes, it is hard to beat Amazon. robert
MGBrose Posted November 21, 2012 Posted November 21, 2012 Amazon EC2 is very good. But it's just a blank machine, no control panels or other admin wizard junk. You'll need to know how to operate a Linux/UNIX system. But as far as providing infrastructure as a service goes, it is hard to beat Amazon. robert When you buy a VPS only buy XEN based vps's, hosts that run OpenVZ tend to oversell whereas Xen based has better resource control. As for where to buy, I've had great experiences with rackspace, while they are pricier with cloudservers, its rock solid. If your looking for a budget dedi: http://volumedrive.com/ is fantastic. We have a dedi with volumedrive that runs us around $100.00 I can't really compliment IPS on their hosting as we got kicked off after being a customer for 9 years, because of the Mass PM bug in their software that cause our CPU usage to spike to huge amounts. So we get kicked for a bug in their own software go figure. . . . Conclusion: Xen VPS or Dedi + WHM = easily managable, easily optimizable setup, that will be cheaper then the cloud and hosted solutions. And often perform as well if not better.
SG Staff Posted January 24, 2013 Author Posted January 24, 2013 Just to update - I took the plunge and moved to a dedicated server. Problem completely solved. Anyone else experiencing these issues might look into a different hosting situation - whether it's moving from Shared to VPS to different company VPS to Dedicated.
Guest Ae9803 Posted March 2, 2013 Posted March 2, 2013 I just ordered a dedicated server from http://volumedrive.com/ so ill be taking the plunge and report back my findings :)
Grumpy Posted March 4, 2013 Posted March 4, 2013 I just ordered a dedicated server from http://volumedrive.com/ so ill be taking the plunge and report back my findings :smile: You may be in for a long wait... volumedrive is known to be as late as month+ after order to actually get deployed...
Guest Ae9803 Posted March 4, 2013 Posted March 4, 2013 I confirmed with them they should have it installed within the next few days.
Bluehell Productions Posted March 19, 2013 Posted March 19, 2013 Past experiences with Volumedrive have been poor :(
Guest Ae9803 Posted March 20, 2013 Posted March 20, 2013 I just got the update that server was online. I wasnt too impressed with the time it took for them to set it up. But after I read reviews, I knew that it could take some time so I wasnt in much of a hurry. After getting it all set up, I was very very very impressed. I couldnt say I can complain, because for the price for the dedicated server it is very worth the wait. I give them an A -, for the service and time. Ill repost in 6 months :D
IveLeft... Posted March 20, 2013 Posted March 20, 2013 When you buy a VPS only buy XEN based vps's, hosts that run OpenVZ tend to oversell whereas Xen based has better resource control. As for where to buy, I've had great experiences with rackspace, while they are pricier with cloudservers, its rock solid. If your looking for a budget dedi: http://volumedrive.com/ is fantastic. We have a dedi with volumedrive that runs us around $100.00 I can't really compliment IPS on their hosting as we got kicked off after being a customer for 9 years, because of the Mass PM bug in their software that cause our CPU usage to spike to huge amounts. So we get kicked for a bug in their own software go figure. . . . Conclusion: Xen VPS or Dedi + WHM = easily managable, easily optimizable setup, that will be cheaper then the cloud and hosted solutions. And often perform as well if not better. KVM is also superb
Recommended Posts
Archived
This topic is now archived and is closed to further replies.