interesting high load on apache server
We are experiencing an unusal high load on our web/db server. It is requesting apache restart nearly every 20 minutes after it goes to load of 100+ . We had checked below
No flood or abusive attack seem during this activity
messages, error_log does not output anything unexpected
Kernel is 2.6.10-1.771_FC2smp
We have upgraded to 2.6.11.6 , Patched with security appliances, does not differ.
Some stat views from now:
httpd status:
Current Time: Friday, 08-Apr-2005 02:25:58 EEST
Restart Time: Friday, 08-Apr-2005 01:51:32 EEST
Parent Server Generation: 0
Server uptime: 34 minutes 26 seconds
Total accesses: 9002 - Total Traffic: 19.4 MB
CPU Usage: u183.98 s68.81 cu.53 cs1.69 - 12.3% CPU load
4.36 requests/sec - 9.6 kB/second - 2265 B/request
25 requests currently being processed, 11 idle servers
top:
top - 02:38:05 up 2:04, 1 user, load average: 8.02, 8.70, 9.03
Tasks: 194 total, 1 running, 192 sleeping, 1 stopped, 0 zombie
Cpu(s): 2.0% us, 1.3% sy, 0.0% ni, 0.0% id, 96.5% wa, 0.2% hi, 0.0% si
Mem: 506128k total, 501976k used, 4152k free, 5196k buffers
Swap: 1048816k total, 180640k used, 868176k free, 109720k cached
mysql status:
Uptime: 7377 Threads: 31 Questions: 239257 Slow queries: 573 Opens: 32669 Flush tables: 1 Open tables: 64 Queries per second avg: 32.433
httpd config:
KeepAlive On
MaxKeepAliveRequests 100
KeepAliveTimeout 5
MinSpareServers 5
MaxSpareServers 15
StartServers 8
MaxClients 150
(* Tried keepalive both off and on. Tried to increase the spare servers)
mysql config:
[mysqld]
port = 3306
socket = /var/tmp/mysql.sock
skip-locking
key_buffer = 256M
max_allowed_packet = 1M
table_cache = 64
sort_buffer_size = 512K
net_buffer_length = 8K
myisam_sort_buffer_size = 8M
wait_timeout = 10
connect_timeout = 10
P4 2.8 GHZ Hyperthread - 512 MB RAM.
What can you advise for our situation?
I noticed we have mod_gzip experimental installed on cpanel. I have uninstalled it to see if that works. could it be the reason?
