MarcusInMd Posted April 11, 2007 Posted April 11, 2007 Thanks guys. We have actually been running 2.2.2 for about a month and a half now. We also have been using eaccelerator for quite a while now. Blair, During these crazy load times I have actually shut down the search capability of our forums much to the dismay of our members. So I know its not search related but something else. I might be willing to give something like httplight a shot, but I will have to research it's setup. Everything I have read so far says to install it on a fresh OS installation.
MarcusInMd Posted April 12, 2007 Posted April 12, 2007 We may actually get another shot at some big numbers now. Computer models are starting to show an insane early spring intense nor'easter snow/rainstorm for the east coast.
njustice Posted April 12, 2007 Posted April 12, 2007 I also like hardware, I currently run a fast Opteron dual core with 4GB of RAM and SCSI drives. I rarely see the load over 1 (too much server :P). Generally 600-700 online during the peak of the day. I've had as many as 2,000 online in the past 15 minutes, and the load hovered around 2. Users won't notice any difference until it gets over 4. For faecess and giggles, here's the current my.cnf (mysqld) and httpd.conf settings:[mysqld] datadir=/var/lib/mysql socket=/var/lib/mysql/mysql.sock skip-locking skip-innodb query_cache_limit=16M query_cache_size=32M query_cache_type=1 interactive_timeout=1200 wait_timeout=100 connect_timeout=20 key_buffer=800M key_cache_division_limit=40 low_priority_updates=1 join_buffer_size=2M record_buffer=6M max_allowed_packet=16M table_cache=1280 tmp_table_size=64M sort_buffer_size=200M key_buffer_size=200M read_buffer_size=2M thread_cache_size=200 max_user_connections=500 max_connections=500 max_connect_errors=10 thread_concurrency=4 I strongly recommend the tuning-primer.sh script to optimise your SQL settings. httpd.conf... Timeout 300 KeepAlive On MaxKeepAliveRequests 100 KeepAliveTimeout 15 MinSpareServers 5 MaxSpareServers 10 StartServers 5 MaxClients 150 MaxRequestsPerChild 0 BTW, I'm running CentOS 64-bit. I'm NOT running a PHP cache program yet, as I haven't found a 64-bit version I'm confident in.
MarcusInMd Posted April 12, 2007 Posted April 12, 2007 Its gotta be hardware related then. I must have an issue somewhere. Its either the RAID SATA drives on the web server or the processor is dying somehow. We have two AMD opteron 242s running on this puppy with 4gb of ram.
njustice Posted April 12, 2007 Posted April 12, 2007 We operate pretty similar hardware on similar sized sites. Why not give my setting a try and see what happens? IF it helps, you can tweak later.
.CMANNS Posted April 12, 2007 Posted April 12, 2007 Thanks guys. We have actually been running 2.2.2 for about a month and a half now. We also have been using eaccelerator for quite a while now. Blair, During these crazy load times I have actually shut down the search capability of our forums much to the dismay of our members. So I know its not search related but something else. I might be willing to give something like httplight a shot, but I will have to research it's setup. Everything I have read so far says to install it on a fresh OS installation. Why a fresh OS setup? I've never read something stating that
kmf Posted April 12, 2007 Posted April 12, 2007 how about you do some good monitoring first, before you point fingers. Login to mysql and take a look on the current running sql-queries with SHOW PROCESSLIST and make conclusions from that Besides that, take a look on the webserver(s) and see what they're doing during peak time. I have 2 webservers myself, taking care of the PHP-codig, one database server and a seperate very cheap VIA-based fileswebserver which host our static files like images, js, css, etc with lighttpd. One of the webservers is also running a seperate search engine for the board, xapian. This serves 3000 people online. Granted, 50% are from the other subsites which shares the same user/sessiontable as the forum and the subsites are custombuilt for speed, but still.... the forum itself is still generating 1500 people online constantly without any hard changes.
njustice Posted April 12, 2007 Posted April 12, 2007 Well, I am about to now drop the table indexes and recreate them. <sigh> Did you ever drop and rebuild your indexes?
MarcusInMd Posted April 12, 2007 Posted April 12, 2007 Yeah, I dropped all indexes and recreated them. I have been monitoring all the processes for 2 years now...This year especially. There are some slow sql queries as I posted above (this was before we got the new DB server) I am about out of options with regards to pointing fingers at the cause of the problem. We have been getting a decent amount of traffic because of this late season storm: 375 user(s) active in the past 15 minutes Server load is 1.15 at the moment. Here is a quick snap shot now: top - 13:53:22 up 46 days, 18:07, 1 user, load average: 1.78, 1.34, 1.14 Tasks: 162 total, 2 running, 160 sleeping, 0 stopped, 0 zombie Cpu(s): 18.1% us, 5.3% sy, 0.0% ni, 75.7% id, 0.0% wa, 0.0% hi, 0.8% si Mem: 3994580k total, 2993612k used, 1000968k free, 188908k buffers Swap: 2031608k total, 772k used, 2030836k free, 1413336k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 18055 apache 15 0 205m 41m 16m S 18.9 1.1 0:42.17 httpd 17345 apache 15 0 206m 44m 17m S 8.3 1.1 1:10.28 httpd 19258 apache 15 0 206m 38m 12m S 5.6 1.0 0:16.32 httpd 19737 apache 16 0 205m 35m 10m R 4.3 0.9 0:02.40 httpd 17521 apache 15 0 206m 42m 16m S 3.3 1.1 0:56.32 httpd 18267 apache 15 0 205m 42m 17m S 3.3 1.1 0:38.72 httpd 17788 apache 15 0 205m 42m 16m S 1.3 1.1 0:51.96 httpd 18868 apache 15 0 206m 43m 17m S 1.3 1.1 0:24.59 httpd 1815 named 16 0 75940 18m 1944 S 0.3 0.5 65:48.17 named 16483 apache 15 0 208m 45m 17m S 0.3 1.2 1:22.62 httpd 17790 apache 15 0 206m 42m 15m S 0.3 1.1 0:50.66 httpd 19026 apache 15 0 205m 39m 14m S 0.3 1.0 0:16.36 httpd 19135 apache 15 0 205m 36m 11m S 0.3 0.9 0:15.77 httpd 19370 apache 15 0 205m 36m 11m S 0.3 0.9 0:08.45 httpd 19841 root 16 0 7432 1120 808 R 0.3 0.0 0:00.03 top 1 root 16 0 4876 596 500 S 0.0 0.0 0:02.09 init 2 root RT 0 0 0 0 S 0.0 0.0 0:01.96 migration/0 DB Server: top - 13:53:44 up 46 days, 14:42, 3 users, load average: 0.27, 0.38, 0.44 Tasks: 124 total, 1 running, 122 sleeping, 0 stopped, 1 zombie Cpu(s): 2.7% us, 0.3% sy, 0.0% ni, 95.3% id, 1.6% wa, 0.0% hi, 0.1% si Mem: 4040568k total, 4003520k used, 37048k free, 79888k buffers Swap: 2031608k total, 160k used, 2031448k free, 3046276k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4287 root 16 0 54240 23m 3236 S 21 0.6 11117:52 X 13437 mysql 15 0 823m 513m 4416 S 3 13.0 5658:10 mysqld 466 root 15 0 0 0 0 S 0 0.0 21:08.92 kjournald 4727 root 16 0 286m 48m 20m S 0 1.2 19:18.40 firefox-bin 24656 root 16 0 6148 1032 776 R 0 0.0 0:00.03 top 1 root 16 0 4752 556 460 S 0 0.0 0:01.34 init 2 root RT 0 0 0 0 S 0 0.0 0:00.08 migration/0 3 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/0 4 root RT 0 0 0 0 S 0 0.0 0:00.09 migration/1 5 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/1 6 root RT 0 0 0 0 S 0 0.0 0:00.07 migration/2 7 root 34 19 0 0 0 S 0 0.0 0:00.01 ksoftirqd/2 8 root RT 0 0 0 0 S 0 0.0 0:00.07 migration/3 9 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/3 10 root RT 0 0 0 0 S 0 0.0 0:00.06 migration/4 11 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/4 12 root RT 0 0 0 0 S 0 0.0 0:00.04 migration/5
.CMANNS Posted April 12, 2007 Posted April 12, 2007 If you had lighttpd your load would be about .30 Its not your DB at all as far as I can tell, your db server isnt used at all whatsoever Whats with the firefox & x though? I'm putting 20 bucks on it being apache
MarcusInMd Posted April 12, 2007 Posted April 12, 2007 When I set the DB box up at the data center I had gnome running on it with firefox to do some stuff. Forgot to shut it down but that server is not even being touched load wise so I have not bothered shutting it down. I am going to the DC tomorrow to swap out one of the raptor drives that died. 350 users on now top - 14:14:58 up 46 days, 18:28, 1 user, load average: 0.89, 0.87, 0.84 Tasks: 166 total, 17 running, 149 sleeping, 0 stopped, 0 zombie Cpu(s): 30.3% us, 8.3% sy, 0.0% ni, 53.6% id, 5.8% wa, 0.3% hi, 1.7% si Mem: 3994580k total, 3031424k used, 963156k free, 189000k buffers Swap: 2031608k total, 772k used, 2030836k free, 1450036k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 21010 apache 15 0 205m 33m 8916 R 17.2 0.9 0:01.75 httpd 20662 apache 15 0 205m 37m 11m S 9.9 1.0 0:11.21 httpd 19135 apache 15 0 205m 40m 15m R 8.3 1.1 0:48.50 httpd 20706 apache 15 0 205m 35m 10m R 7.6 0.9 0:07.84 httpd 18445 popuser 16 0 122m 47m 2432 R 7.0 1.2 0:51.45 spamd 20711 apache 16 0 206m 36m 10m S 7.0 0.9 0:07.75 httpd 20130 apache 15 0 205m 39m 13m R 4.6 1.0 0:25.56 httpd 18618 apache 15 0 206m 42m 16m R 2.0 1.1 0:54.84 httpd 20665 apache 15 0 205m 37m 12m S 1.7 1.0 0:08.49 httpd 1815 named 16 0 75940 18m 1944 S 1.3 0.5 65:50.35 named 20136 apache 15 0 205m 39m 14m S 1.3 1.0 0:24.48 httpd 20671 apache 15 0 205m 37m 12m R 1.3 1.0 0:10.10 httpd 20876 apache 15 0 205m 36m 11m S 1.3 0.9 0:05.05 httpd 17345 apache 15 0 207m 46m 19m R 1.0 1.2 1:46.26 httpd 19258 apache 15 0 206m 41m 15m S 1.0 1.1 0:47.68 httpd 19370 apache 15 0 205m 41m 15m R 1.0 1.1 0:42.00 httpd 19863 drweb 16 0 19652 13m 812 S 1.0 0.3 0:00.12 drwebd 20579 apache 15 0 205m 36m 11m S 1.0 0.9 0:09.20 httpd 20660 apache 15 0 205m 35m 10m R 0.7 0.9 0:06.90 httpd 17065 apache 15 0 208m 46m 18m S 0.3 1.2 1:47.87 httpd 18868 apache 15 0 206m 44m 18m R 0.3 1.1 0:57.33 httpd 19273 apache 15 0 206m 41m 16m R 0.3 1.1 0:44.29 httpd 20093 apache 15 0 205m 39m 14m R 0.3 1.0 0:26.40 httpd 20666 apache 15 0 205m 37m 12m R 0.3 1.0 0:08.53 httpd 1 root 16 0 4876 596 500 S 0.0 0.0 0:02.09 init 2 root RT 0 0 0 0 S 0.0 0.0 0:01.96 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:01.09 ksoftirqd/0 4 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0 5 root RT 0 0 0 0 S 0.0 0.0 0:25.11 migration/1 6 root 34 19 0 0 0 S 0.0 0.0 0:00.55 ksoftirqd/1 7 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/1 8 root 10 -5 0 0 0 S 0.0 0.0 0:00.64 events/0 9 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 events/1 10 root 20 -5 0 0 0 S 0.0 0.0 0:00.00 khelper 11 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kthread 16 root 10 -5 0 0 0 S 0.0 0.0 0:05.10 kblockd/0 17 root 10 -5 0 0 0 S 0.0 0.0 0:00.51 kblockd/1 18 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid 110 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khubd
MarcusInMd Posted April 12, 2007 Posted April 12, 2007 437 user(s) active in the past 15 minutes Anything I should be looking at right now? top - 15:41:57 up 46 days, 19:55, 1 user, load average: 2.83, 2.21, 1.83 Tasks: 173 total, 3 running, 170 sleeping, 0 stopped, 0 zombie Cpu(s): 26.8% us, 10.0% sy, 0.0% ni, 62.1% id, 0.0% wa, 0.0% hi, 1.2% si Mem: 3994580k total, 3291744k used, 702836k free, 189484k buffers Swap: 2031608k total, 772k used, 2030836k free, 1653952k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 24432 apache 16 0 206m 43m 17m S 10.3 1.1 1:01.86 httpd 26119 apache 15 0 205m 39m 14m S 10.0 1.0 0:27.77 httpd 26216 apache 15 0 205m 37m 12m S 9.6 1.0 0:22.81 httpd 25988 apache 15 0 205m 40m 15m S 9.3 1.0 0:22.16 httpd 26537 apache 15 0 205m 37m 12m S 9.3 1.0 0:12.92 httpd 26471 apache 15 0 205m 38m 13m S 7.0 1.0 0:13.69 httpd 27023 apache 16 0 205m 31m 7812 S 3.3 0.8 0:00.10 httpd 26248 apache 15 0 206m 39m 13m S 2.7 1.0 0:22.34 httpd 25942 apache 17 0 205m 39m 14m S 2.0 1.0 0:30.58 httpd 26214 apache 15 0 206m 39m 13m S 2.0 1.0 0:18.61 httpd 26962 apache 15 0 206m 35m 9.8m R 1.3 0.9 0:01.12 httpd 25949 apache 15 0 206m 41m 15m S 1.0 1.1 0:28.07 httpd 25976 apache 15 0 206m 39m 13m S 1.0 1.0 0:22.33 httpd 26843 apache 15 0 205m 35m 9968 S 1.0 0.9 0:04.14 httpd 26757 apache 15 0 205m 38m 13m S 0.7 1.0 0:06.96 httpd 2674 root 15 0 40840 2568 1960 S 0.3 0.1 5:01.42 monit 20050 root 16 0 7428 1144 812 R 0.3 0.0 0:18.07 top 26120 apache 16 0 206m 40m 15m S 0.3 1.0 0:25.90 httpd 26479 apache 15 0 206m 38m 12m S 0.3 1.0 0:16.81 httpd 27021 apache 15 0 205m 31m 7664 S 0.3 0.8 0:00.30 httpd 1 root 16 0 4876 596 500 S 0.0 0.0 0:02.09 init 2 root RT 0 0 0 0 S 0.0 0.0 0:01.96 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:01.09 ksoftirqd/0 4 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0 5 root RT 0 0 0 0 S 0.0 0.0 0:25.28 migration/1 6 root 34 19 0 0 0 S 0.0 0.0 0:00.56 ksoftirqd/1 7 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/1 8 root 10 -5 0 0 0 S 0.0 0.0 0:00.64 events/0 DB Server: top - 15:42:37 up 46 days, 16:31, 3 users, load average: 0.56, 0.57, 0.57 Tasks: 124 total, 1 running, 122 sleeping, 0 stopped, 1 zombie Cpu(s): 3.3% us, 0.4% sy, 0.0% ni, 93.3% id, 2.8% wa, 0.0% hi, 0.2% si Mem: 4040568k total, 4007048k used, 33520k free, 79892k buffers Swap: 2031608k total, 160k used, 2031448k free, 3046816k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 4287 root 16 0 54240 23m 3236 S 21 0.6 11135:33 X 13437 mysql 15 0 823m 514m 4416 S 9 13.0 5695:58 mysqld 466 root 15 0 0 0 0 S 0 0.0 21:16.89 kjournald 4727 root 15 0 286m 48m 20m S 0 1.2 19:20.38 firefox-bin
MarcusInMd Posted April 12, 2007 Posted April 12, 2007 Here is a link to our forum in case this storm interests anyone here or you would like to see whats going on with the software:http://www.easternuswx.com/bb/index.php?act=idx
MarcusInMd Posted April 12, 2007 Posted April 12, 2007 Our httpd.conf : # # Timeout: The number of seconds before receives and sends time out. # Timeout 120 # # KeepAlive: Whether or not to allow persistent connections (more than # one request per connection). Set to "Off" to deactivate. # KeepAlive Off # # MaxKeepAliveRequests: The maximum number of requests to allow # during a persistent connection. Set to 0 to allow an unlimited amount. # We recommend you leave this number high, for maximum performance. # MaxKeepAliveRequests 100 # # KeepAliveTimeout: Number of seconds to wait for the next request from the # same client on the same connection. # KeepAliveTimeout 2 ## ## Server-Pool Size Regulation (MPM specific) ## # prefork MPM # StartServers: number of server processes to start # MinSpareServers: minimum number of server processes which are kept spare # MaxSpareServers: maximum number of server processes which are kept spare # ServerLimit: maximum value for MaxClients for the lifetime of the server # MaxClients: maximum number of server processes allowed to start # MaxRequestsPerChild: maximum number of requests a server process serves <IfModule prefork.c> #StartServers 8 #MinSpareServers 5 #MaxSpareServers 20 #ServerLimit 256 #MaxClients 256 #MaxRequestsPerChild 4000 ##StartServers 8 StartServers 5 MinSpareServers 5 MaxSpareServers 20 ServerLimit 256 MaxClients 125 #MaxRequestsPerChild 1000 MaxRequestsPerChild 0 </IfModule> # worker MPM # StartServers: initial number of server processes to start # MaxClients: maximum number of simultaneous client connections # MinSpareThreads: minimum number of worker threads which are kept spare # MaxSpareThreads: maximum number of worker threads which are kept spare # ThreadsPerChild: constant number of worker threads in each server process # MaxRequestsPerChild: maximum number of requests a server process serves <IfModule worker.c> #StartServers 2 #MaxClients 150 #MinSpareThreads 25 #MaxSpareThreads 75 #ThreadsPerChild 25 #MaxRequestsPerChild 0 ##StartServers 2 #StartServers 3 #MaxClients 125 #MinSpareThreads 25 #MaxSpareThreads 75 #ThreadsPerChild 25 ##MaxRequestsPerChild 1000 #MaxRequestsPerChild 0 </IfModule> # # Listen: Allows you to bind Apache to specific IP addresses and/or # ports, in addition to the default. See also the <VirtualHost> # directive. # # Change this to Listen on specific IP addresses as shown below to # prevent Apache from glomming onto all bound IP addresses (0.0.0.0) # #Listen 12.34.56.78:80 Listen 80 # # Dynamic Shared Object (DSO) Support # # To be able to use the functionality of a module which was built as a DSO you # have to place corresponding `LoadModule' lines at this location so the # directives contained in it are actually available _before_ they are used. # Statically compiled modules (those listed by `httpd -l') do not need # to be loaded here. # # Example: # LoadModule foo_module modules/mod_foo.so # LoadModule access_module modules/mod_access.so LoadModule auth_module modules/mod_auth.so LoadModule auth_anon_module modules/mod_auth_anon.so LoadModule auth_dbm_module modules/mod_auth_dbm.so LoadModule auth_digest_module modules/mod_auth_digest.so LoadModule ldap_module modules/mod_ldap.so LoadModule auth_ldap_module modules/mod_auth_ldap.so LoadModule include_module modules/mod_include.so LoadModule log_config_module modules/mod_log_config.so LoadModule logio_module modules/mod_logio.so LoadModule env_module modules/mod_env.so #LoadModule mime_magic_module modules/mod_mime_magic.so LoadModule cern_meta_module modules/mod_cern_meta.so LoadModule expires_module modules/mod_expires.so LoadModule deflate_module modules/mod_deflate.so LoadModule headers_module modules/mod_headers.so LoadModule usertrack_module modules/mod_usertrack.so LoadModule setenvif_module modules/mod_setenvif.so LoadModule mime_module modules/mod_mime.so LoadModule dav_module modules/mod_dav.so LoadModule status_module modules/mod_status.so LoadModule autoindex_module modules/mod_autoindex.so LoadModule asis_module modules/mod_asis.so LoadModule info_module modules/mod_info.so LoadModule dav_fs_module modules/mod_dav_fs.so LoadModule vhost_alias_module modules/mod_vhost_alias.so LoadModule negotiation_module modules/mod_negotiation.so LoadModule dir_module modules/mod_dir.so LoadModule actions_module modules/mod_actions.so LoadModule speling_module modules/mod_speling.so LoadModule userdir_module modules/mod_userdir.so LoadModule alias_module modules/mod_alias.so LoadModule rewrite_module modules/mod_rewrite.so LoadModule proxy_module modules/mod_proxy.so LoadModule proxy_ftp_module modules/mod_proxy_ftp.so LoadModule proxy_http_module modules/mod_proxy_http.so LoadModule proxy_connect_module modules/mod_proxy_connect.so LoadModule cache_module modules/mod_cache.so LoadModule suexec_module modules/mod_suexec.so LoadModule disk_cache_module modules/mod_disk_cache.so LoadModule file_cache_module modules/mod_file_cache.so LoadModule mem_cache_module modules/mod_mem_cache.so LoadModule cgi_module modules/mod_cgi.so LoadModule frontpage_module /usr/lib/httpd/modules/mod_frontpage.so IPB Version 2.2.2 (ID:) MYSQL Version MYSQL 4.1.20-log PHP Version 5.0.4 (apache2handler) ( PHP INFO ) Disabled PHP Functions no information Safe Mode OFF Server Software Linux 2.6.17-1.2142_FC4smp #1 SMP Tue Jul 11 22:59:20 EDT 2006 x86_64 Current Server Load 1.71 Total Server Memory 3900 MB Available Physical Memory 643 MB I am a bit under the weather at the moment, how does one get the apache version? I think its 2.0 something or another.
rct2·com Posted April 12, 2007 Posted April 12, 2007 I am a bit under the weather at the moment, ... Seems appropriate ;)httpd -ver
MarcusInMd Posted April 13, 2007 Posted April 13, 2007 Here we go again 423 user(s) active in the past 15 minutes top - 22:45:58 up 47 days, 2:59, 1 user, load average: 13.55, 6.45, 4.06 Tasks: 202 total, 11 running, 191 sleeping, 0 stopped, 0 zombie Cpu(s): 58.5% us, 15.1% sy, 0.0% ni, 21.7% id, 0.0% wa, 0.0% hi, 4.7% si Mem: 3994580k total, 3962244k used, 32336k free, 164900k buffers Swap: 2031608k total, 6812k used, 2024796k free, 2334996k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 19056 apache 16 0 206m 35m 10m R 28.1 0.9 0:05.12 httpd 19308 apache 16 0 206m 34m 8784 R 24.4 0.9 0:00.71 httpd 18208 apache 15 0 206m 39m 13m S 16.9 1.0 0:25.33 httpd 19144 apache 15 0 206m 36m 10m S 15.0 0.9 0:04.02 httpd 19027 apache 16 0 206m 35m 10m R 13.1 0.9 0:04.70 httpd 19239 apache 15 0 207m 35m 9448 S 13.1 0.9 0:01.59 httpd 19133 apache 15 0 205m 34m 9048 S 5.6 0.9 0:03.84 httpd 19207 apache 16 0 205m 33m 8944 R 5.6 0.9 0:02.05 httpd 18632 apache 15 0 206m 38m 12m S 3.8 1.0 0:17.22 httpd 19029 apache 15 0 205m 34m 9636 S 3.8 0.9 0:07.26 httpd 19059 apache 15 0 206m 35m 10m R 3.8 0.9 0:04.07 httpd 19094 apache 15 0 206m 37m 12m S 3.8 1.0 0:05.48 httpd 13181 root 15 0 7428 1156 812 R 1.9 0.0 0:18.97 top 18819 apache 15 0 205m 37m 12m S 1.9 1.0 0:11.93 httpd 18991 apache 15 0 206m 36m 10m R 1.9 0.9 0:05.70 httpd 19268 apache 15 0 205m 33m 8312 S 1.9 0.8 0:01.02 httpd 19272 apache 15 0 201m 22m 1752 S 1.9 0.6 0:00.09 httpd 19273 apache 15 0 203m 27m 5148 S 1.9 0.7 0:00.18 httpd 19274 apache 15 0 205m 32m 7940 S 1.9 0.8 0:00.67 httpd 1 root 15 0 4876 596 500 S 0.0 0.0 0:02.11 init 2 root RT 0 0 0 0 S 0.0 0.0 0:01.97 migration/0 3 root 34 19 0 0 0 S 0.0 0.0 0:01.09 ksoftirqd/0 4 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/0 5 root RT 0 0 0 0 S 0.0 0.0 0:25.92 migration/1 6 root 34 19 0 0 0 S 0.0 0.0 0:00.56 ksoftirqd/1 7 root RT 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/1 8 root 10 -5 0 0 0 S 0.0 0.0 0:00.64 events/0 9 root 10 -5 0 0 0 S 0.0 0.0 0:00.01 events/1 10 root 20 -5 0 0 0 S 0.0 0.0 0:00.00 khelper 11 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kthread 16 root 10 -5 0 0 0 S 0.0 0.0 0:05.15 kblockd/0 17 root 10 -5 0 0 0 S 0.0 0.0 0:00.52 kblockd/1 18 root 12 -5 0 0 0 S 0.0 0.0 0:00.00 kacpid 110 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 khubd 112 root 10 -5 0 0 0 S 0.0 0.0 0:00.00 kseriod 178 root 15 0 0 0 0 S 0.0 0.0 0:07.60 kswapd0 DB Server: top - 22:46:38 up 46 days, 23:35, 3 users, load average: 1.25, 0.96, 0.86 Tasks: 124 total, 2 running, 121 sleeping, 0 stopped, 1 zombie Cpu(s): 5.6% us, 1.4% sy, 0.0% ni, 90.9% id, 1.8% wa, 0.0% hi, 0.2% si Mem: 4040568k total, 4015400k used, 25168k free, 79948k buffers Swap: 2031608k total, 160k used, 2031448k free, 3049072k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 13437 mysql 15 0 823m 518m 4416 S 32 13.1 5836:05 mysqld 4287 root 16 0 54372 23m 3236 S 24 0.6 11205:13 X 466 root 15 0 0 0 0 D 0 0.0 21:47.43 kjournald 1 root 16 0 4752 556 460 S 0 0.0 0:01.34 init 2 root RT 0 0 0 0 S 0 0.0 0:00.08 migration/0 3 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/0 4 root RT 0 0 0 0 S 0 0.0 0:00.09 migration/1 5 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/1 6 root RT 0 0 0 0 S 0 0.0 0:00.07 migration/2 7 root 34 19 0 0 0 S 0 0.0 0:00.01 ksoftirqd/2 8 root RT 0 0 0 0 S 0 0.0 0:00.07 migration/3 9 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/3 10 root RT 0 0 0 0 S 0 0.0 0:00.06 migration/4 11 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/4 12 root RT 0 0 0 0 S 0 0.0 0:00.04 migration/5 13 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/5 14 root RT 0 0 0 0 S 0 0.0 0:00.02 migration/6 15 root 34 19 0 0 0 S 0 0.0 0:00.02 ksoftirqd/6 16 root RT 0 0 0 0 S 0 0.0 0:00.00 migration/7 17 root 34 19 0 0 0 S 0 0.0 0:00.00 ksoftirqd/7 18 root 5 -10 0 0 0 S 0 0.0 0:00.00 events/0 19 root 5 -10 0 0 0 S 0 0.0 0:00.00 events/1 20 root 5 -10 0 0 0 S 0 0.0 0:00.00 events/2 21 root 5 -10 0 0 0 S 0 0.0 0:00.00 events/3 22 root 5 -10 0 0 0 S 0 0.0 0:00.00 events/4 23 root 5 -10 0 0 0 S 0 0.0 0:00.00 events/5 24 root 5 -10 0 0 0 S 0 0.0 0:00.00 events/6 25 root 5 -10 0 0 0 S 0 0.0 0:00.00 events/7 26 root 5 -10 0 0 0 S 0 0.0 0:00.00 khelper 27 root 15 -10 0 0 0 S 0 0.0 0:00.00 kacpid 64 root 5 -10 0 0 0 S 0 0.0 0:00.00 kblockd/0 65 root 5 -10 0 0 0 S 0 0.0 0:00.00 kblockd/1 66 root 5 -10 0 0 0 S 0 0.0 0:00.00 kblockd/2 67 root 5 -10 0 0 0 S 0 0.0 0:00.00 kblockd/3 68 root 5 -10 0 0 0 S 0 0.0 0:00.00 kblockd/4
.CMANNS Posted April 13, 2007 Posted April 13, 2007 Doesn't the "Members Online Today: 709 [Expand][Collapse]" mod cause higher load? PHP Wise I thought it did.
MarcusInMd Posted April 13, 2007 Posted April 13, 2007 Doesn't the "Members Online Today: 709 [Expand][Collapse]" mod cause higher load? PHP Wise I thought it did. I have no idea, but in the past when problems have arised I have shut down just about all of the mods that I have made.
.CMANNS Posted April 13, 2007 Posted April 13, 2007 I have no idea, but in the past when problems have arised I have shut down just about all of the mods that I have made. I'd try that one off anyways just to see if it had a performance hit...
MarcusInMd Posted April 13, 2007 Posted April 13, 2007 I'd try that one off anyways just to see if it had a performance hit... That one was included. Didn't help.
rct2·com Posted April 13, 2007 Posted April 13, 2007 Here we go again 423 user(s) active in the past 15 minutes top - 22:45:58 up 47 days, 2:59, 1 user, load average: 13.55, 6.45, 4.06 Tasks: 202 total, 11 running, 191 sleeping, 0 stopped, 0 zombie Cpu(s): 58.5% us, 15.1% sy, 0.0% ni, 21.7% id, 0.0% wa, 0.0% hi, 4.7% si Mem: 3994580k total, 3962244k used, 32336k free, 164900k buffers Swap: 2031608k total, 6812k used, 2024796k free, 2334996k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 19056 apache 16 0 206m 35m 10m R 28.1 0.9 0:05.12 httpd 19308 apache 16 0 206m 34m 8784 R 24.4 0.9 0:00.71 httpd 18208 apache 15 0 206m 39m 13m S 16.9 1.0 0:25.33 httpd 19144 apache 15 0 206m 36m 10m S 15.0 0.9 0:04.02 httpd 19027 apache 16 0 206m 35m 10m R 13.1 0.9 0:04.70 httpd 19239 apache 15 0 207m 35m 9448 S 13.1 0.9 0:01.59 httpd 19133 apache 15 0 205m 34m 9048 S 5.6 0.9 0:03.84 httpd 19207 apache 16 0 205m 33m 8944 R 5.6 0.9 0:02.05 httpd 18632 apache 15 0 206m 38m 12m S 3.8 1.0 0:17.22 httpd 19029 apache 15 0 205m 34m 9636 S 3.8 0.9 0:07.26 httpd 19059 apache 15 0 206m 35m 10m R 3.8 0.9 0:04.07 httpd 19094 apache 15 0 206m 37m 12m S 3.8 1.0 0:05.48 httpd 13181 root 15 0 7428 1156 812 R 1.9 0.0 0:18.97 top 18819 apache 15 0 205m 37m 12m S 1.9 1.0 0:11.93 httpd 18991 apache 15 0 206m 36m 10m R 1.9 0.9 0:05.70 httpd 19268 apache 15 0 205m 33m 8312 S 1.9 0.8 0:01.02 httpd 19272 apache 15 0 201m 22m 1752 S 1.9 0.6 0:00.09 httpd 19273 apache 15 0 203m 27m 5148 S 1.9 0.7 0:00.18 httpd 19274 apache 15 0 205m 32m 7940 S 1.9 0.8 0:00.67 httpd <snip> DB Server: top - 22:46:38 up 46 days, 23:35, 3 users, load average: 1.25, 0.96, 0.86 Tasks: 124 total, 2 running, 121 sleeping, 0 stopped, 1 zombie Cpu(s): 5.6% us, 1.4% sy, 0.0% ni, 90.9% id, 1.8% wa, 0.0% hi, 0.2% si Mem: 4040568k total, 4015400k used, 25168k free, 79948k buffers Swap: 2031608k total, 160k used, 2031448k free, 3049072k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 13437 mysql 15 0 823m 518m 4416 S 32 13.1 5836:05 mysqld 4287 root 16 0 54372 23m 3236 S 24 0.6 11205:13 X 466 root 15 0 0 0 0 D 0 0.0 21:47.43 kjournald 1 root 16 0 4752 556 460 S 0 0.0 0:01.34 init <snip> The first of these tops illustrates that while only half of your CPU cycles are being used, there are processes queuing up to get CPU but not being allocated any. Second, there are some Apache processes that are taking disproportionate amounts of CPU, all are using insignificant amounts of memory. This suggests that something in Apache (or compiled into Apache) is taking a long time to execute, and not entering a wait state to allow other processes some CPU cycles. I do believe that the next step is to look at your access logs for a 5 minute window around this top listing, and find out what Apache was being asked to do (i.e. what URLS were being hit). PS. Hope you're feeling better today. :)
.CMANNS Posted April 13, 2007 Posted April 13, 2007 rct2dotcom, I'm guessing hes running over 50req/s, or atleast close to it, apache usually looks like that with high req/s, also php is included in each http process if its a php page.
njustice Posted April 13, 2007 Posted April 13, 2007 I'd turn keepailve on. Set it for about 10sec. Also, the members online today has a known problem where it causes high loads. I'd remove it ASAP. Something on this server besides the forum? Seems like a lot of httpd requests.
Recommended Posts
Archived
This topic is now archived and is closed to further replies.