Unusual very high server loads (up to 90!)
These are the server caracteristics:
AMD Sempron 2800 processor
1024MB RAM
80GB HD
100 accounts
CentOS 3.4
Kernel 2.6.11.7
cPanel
APF & BDF installed
turck-mmcache installed
mod_dosevasive.c installed
MySQL 4
PHP Version 4.3.10
Of the 100 accounts all but 2 have very small traffic, generating less than 1gb of server transfer/month. The other two generate about 3gb to 4gb server/transfer monthly and one of them hosts a phpBB forum.
This are the pertinent settings in my.cnf
Code:
query_cache_limit=2M query_cache_size=48M query_cache_type=1 max_connections=400 interactive_timeout=10 wait_timeout=10 connect_timeout=10 thread_cache_size=128 key_buffer=64M join_buffer=1M max_allowed_packet=16M table_cache=1024 record_buffer=2M sort_buffer_size=768K read_buffer_size=512K read_rnd_buffer_size=512K max_connect_errors=10 thread_concurrency=2 myisam_sort_buffer_size=32M log-bin server-id=1 [mysql.server] user=mysql basedir=/var/lib [safe_mysqld] err-log=/var/log/mysqld.log open_files_limit=8192 [mysqldump] quick max_allowed_packet=16M [mysql] no-auto-rehash #safe-updates [isamchk] key_buffer=32M sort_buffer=32M read_buffer=768K write_buffer=16M [myisamchk] key_buffer=32M sort_buffer=32M read_buffer=768K write_buffer=16M [mysqlhotcopy] interactive-timeout
Code:
Timeout 30 KeepAlive Off MinSpareServers 10 MaxSpareServers 20 StartServers 5 MaxClients 300 MaxRequestsPerChild 50
Code:
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.0 1584 504 ? S Apr11 0:16 init [3] root 2 0.0 0.0 0 0 ? SWN Apr11 0:31 [ksoftirqd/0] root 3 0.0 0.0 0 0 ? SW< Apr11 0:12 [events/0] root 4 0.0 0.0 0 0 ? SW< Apr11 0:00 [khelper] root 5 0.0 0.0 0 0 ? SW< Apr11 0:00 [kthread] root 6 0.4 0.0 0 0 ? SW< Apr11 6:24 [kblockd/0] root 7 0.1 0.0 0 0 ? SW Apr11 1:43 [pdflush] root 10 0.0 0.0 0 0 ? SW< Apr11 0:00 [aio/0] root 9 0.3 0.0 0 0 ? SW Apr11 4:09 [kswapd0] root 11 0.0 0.0 0 0 ? SW Apr11 0:00 [kseriod] root 12 0.0 0.0 0 0 ? SW< Apr11 0:00 [ata/0] root 13 0.8 0.0 0 0 ? SW Apr11 11:11 [kjournald] root 358 0.0 0.0 0 0 ? SW Apr11 0:00 [kjournald] root 359 1.1 0.0 0 0 ? SW Apr11 15:31 [kjournald] root 916 0.0 0.0 1636 572 ? S Apr11 0:26 syslogd -m 0 root 920 0.0 0.0 1588 464 ? S Apr11 0:00 klogd -x nobody 956 0.0 0.2 4312 2144 ? SNL Apr11 0:03 proftpd: (accepting connections) root 1938 0.0 0.1 3712 1568 ? S Apr11 0:00 /usr/sbin/sshd root 1954 0.0 0.0 2196 844 ? S Apr11 0:01 xinetd -stayalive -pidfile /var/run/xinetd.pid root 1972 0.0 0.2 6104 2704 ? S Apr11 0:04 chkservd root 2284 0.0 0.0 1652 652 ? S Apr11 0:02 crond nobody 2637 0.0 0.1 3436 2028 ? S Apr11 0:00 entropychat nobody 2646 0.0 0.0 1800 616 ? SN Apr11 0:00 /usr/local/cpanel/bin/startmelange cpanel 2669 0.0 0.1 20252 1960 ? SN Apr11 0:00 /usr/bin/stunnel-4.04local /usr/local/cpanel/etc/stunnel/default/stunnel.conf root 2724 0.0 0.0 1600 504 ? S Apr11 0:00 /usr/sbin/portsentry -tcp root 2870 0.0 0.6 10292 6824 ? S Apr11 0:00 /usr/bin/perl -w /usr/sbin/psad root 2872 0.0 0.0 1568 340 ? S Apr11 0:00 /usr/sbin/kmsgsd root 2874 0.0 0.0 1568 356 ? S Apr11 0:18 /usr/sbin/psadwatchd root 2883 0.0 0.0 1576 424 tty1 S Apr11 0:00 /sbin/mingetty tty1 root 2884 0.0 0.0 1576 424 tty2 S Apr11 0:00 /sbin/mingetty tty2 root 2885 0.0 0.0 1576 424 tty3 S Apr11 0:00 /sbin/mingetty tty3 root 2886 0.0 0.0 1576 424 tty4 S Apr11 0:00 /sbin/mingetty tty4 root 2887 0.0 0.0 1576 424 tty5 S Apr11 0:00 /sbin/mingetty tty5 root 2888 0.0 0.0 1576 424 tty6 S Apr11 0:00 /sbin/mingetty tty6 root 26549 0.1 0.0 0 0 ? SW Apr11 1:27 [pdflush] root 30501 0.0 0.0 0 0 ? SW< Apr11 0:54 [loop0] root 30502 0.0 0.0 0 0 ? SW Apr11 0:19 [kjournald] root 8395 0.0 0.0 1656 660 ? S 01:00 0:00 CROND root 8396 0.0 0.4 5800 4424 ? SN 01:00 0:00 /usr/bin/perl /scripts/cpbackup mailnull 9458 0.0 0.0 0 0 ? ZN 01:01 0:00 [exim <defunct>] alt1040 22087 0.0 0.2 3792 2348 ? S 12:12 0:04 imapd root 30105 0.0 0.2 7056 2312 ? S 12:19 0:02 sshd: root@pts/0 root 30121 0.0 0.2 6904 2164 ? S 12:19 0:02 sshd: root@pts/1 root 30155 0.0 0.1 4304 1388 pts/0 S 12:19 0:03 -bash root 1651 0.0 0.1 4300 1356 pts/1 S 12:20 0:00 -bash root 19652 0.0 0.0 1568 380 ? SN 14:56 0:02 /usr/local/cpanel/bin/cpuwatch 1.0 /scripts/pkgacct mibitaco /backups/cpbackup/ root 19653 0.0 0.2 3692 2328 ? TN 14:56 0:00 pkgacct - mibitaco root 21680 0.1 0.0 0 0 ? ZN 15:28 0:03 [tar <defunct>] mibitaco 22125 0.0 0.1 2688 1292 ? S 15:31 0:01 imapd phicom 23557 0.2 0.1 3040 1580 ? S 15:32 0:05 imapd root 29491 0.3 0.1 4336 1308 pts/1 S 15:33 0:06 top mailnull 30523 0.0 0.1 6664 1884 ? S 15:35 0:00 /usr/sbin/exim -bd -q60m mailnull 30527 0.0 0.1 6664 1872 ? S 15:35 0:00 /usr/sbin/exim -bd -oX 26 mailnull 30531 0.0 0.1 6664 1844 ? S 15:35 0:00 /usr/sbin/exim -tls-on-connect -bd -oX 465 root 30555 0.0 1.9 21888 19984 ? SN 15:35 0:00 /usr/bin/spamd -d --allowed-ips=127.0.0.1 --pidfile=/var/run/spamd.pid --max-ch root 30576 0.3 0.1 2968 1436 ? S 15:35 0:06 antirelayd root 30586 0.4 2.2 24396 22572 ? SN 15:35 0:08 spamd child root 30587 0.1 2.1 23392 21592 ? SN 15:35 0:03 spamd child root 30588 0.2 2.2 24452 22648 ? SN 15:35 0:04 spamd child root 30589 0.1 2.1 23524 21720 ? SN 15:35 0:03 spamd child root 30590 0.1 2.2 24284 22460 ? SN 15:35 0:04 spamd child named 1859 0.3 0.3 31024 3988 ? S 15:38 0:06 /usr/sbin/named -u named root 2335 0.1 0.6 43568 7124 ? SN 15:39 0:02 /usr/local/apache/bin/httpd -DSSL vxcom 2336 0.4 0.1 2736 1320 ? S 15:39 0:07 imapd root 3749 0.0 0.0 2180 980 ? S 15:41 0:00 /bin/sh /usr/bin/mysqld_safe --datadir=/var/lib/mysql --pid-file=/var/lib/mysql mysql 3999 0.0 3.6 127888 36884 ? SN 15:41 0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 4359 0.0 3.6 127888 36884 ? SN 15:41 0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 4360 0.0 3.6 127888 36884 ? SN 15:41 0:00 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 4753 0.8 3.6 127888 36884 ? SN 15:41 0:13 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 4754 0.5 3.6 127888 36884 ? SN 15:41 0:09 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 4755 0.7 3.6 127888 36884 ? SN 15:41 0:12 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 5061 0.6 3.6 127888 36884 ? SN 15:41 0:10 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 5206 0.7 3.6 127888 36884 ? SN 15:41 0:12 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 5211 0.7 3.6 127888 36884 ? SN 15:41 0:12 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 5226 0.5 3.6 127888 36884 ? SN 15:41 0:09 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 5234 0.6 3.6 127888 36884 ? SN 15:41 0:11 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 5252 0.6 3.6 127888 36884 ? SN 15:41 0:10 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 5254 0.4 3.6 127888 36884 ? SN 15:41 0:08 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 5267 0.7 3.6 127888 36884 ? SN 15:41 0:12 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 5276 0.6 3.6 127888 36884 ? SN 15:41 0:11 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ bitacora 5411 0.0 0.0 1436 320 ? SN 15:41 0:00 /usr/local/cpanel/bin/logrunner 1.0 /usr/local/cpanel/3rdparty/bin/awstats.pl - bitacora 5412 0.0 0.0 1024 12 ? TN 15:41 0:00 /usr/bin/perl /usr/local/cpanel/3rdparty/bin/awstats.pl -config=bitacoras.net - mysql 11610 0.5 3.6 127888 36884 ? SN 15:48 0:06 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 11611 0.5 3.6 127888 36884 ? SN 15:48 0:07 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 11612 0.3 3.6 127888 36884 ? SN 15:48 0:04 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 16727 0.3 3.6 127888 36884 ? SN 15:52 0:03 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 16753 0.4 3.6 127888 36884 ? SN 15:53 0:04 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 16754 0.4 3.6 127888 36884 ? SN 15:53 0:04 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 16755 0.2 3.6 127888 36884 ? SN 15:53 0:02 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 16857 0.3 3.6 127888 36884 ? SN 15:53 0:02 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 16858 0.3 3.6 127888 36884 ? SN 15:53 0:03 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 16859 0.3 3.6 127888 36884 ? RN 15:53 0:03 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ mysql 16860 0.5 3.6 127888 36884 ? SN 15:53 0:05 /usr/sbin/mysqld --basedir=/ --datadir=/var/lib/mysql --user=mysql --pid-file=/ nobody 17479 0.0 1.0 44060 11188 ? SN 15:55 0:00 /usr/local/apache/bin/httpd -DSSL mailman 20116 0.0 0.4 7232 4776 ? SN 15:58 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/mailmanctl -s start mailman 20123 0.0 0.4 7184 4700 ? SN 15:58 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=ArchRu mailman 20124 0.0 0.4 7148 4712 ? SN 15:58 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=Bounce mailman 20125 0.0 0.4 7144 4704 ? SN 15:58 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=Comman mailman 20126 0.0 0.4 7224 4700 ? SN 15:58 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=Incomi mailman 20127 0.0 0.4 7224 4740 ? SN 15:58 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=NewsRu mailman 20134 0.0 0.4 7284 4784 ? SN 15:58 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=Outgoi mailman 20135 0.0 0.4 7180 4696 ? SN 15:58 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=Virgin mailman 20136 0.0 0.4 7180 4696 ? SN 15:58 0:00 /usr/bin/python2 /usr/local/cpanel/3rdparty/mailman/bin/qrunner --runner=RetryR root 20154 0.0 0.5 7212 5576 ? SN 15:58 0:00 cpanellogd - sleeping for logs root 20158 0.0 0.6 8228 6724 ? SN 15:58 0:00 cppop - accepting on port 110 root 20162 0.0 0.7 9584 8064 ? S 15:58 0:00 cpsrvd - waiting for connections nobody 23535 0.0 0.7 43724 7652 ? SN 16:00 0:00 /usr/local/apache/bin/httpd -DSSL datagram 24400 0.0 0.2 4964 2856 ? SNL 16:01 0:00 proftpd: datagram - 200.69.190.106: IDLE nobody 24687 0.2 1.2 44376 12292 ? SN 16:03 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27402 0.2 0.0 0 0 ? ZN 16:06 0:00 [httpd <defunct>] nobody 27425 0.4 1.4 45000 14396 ? SN 16:07 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27449 0.5 1.3 44788 14204 ? SN 16:07 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27453 0.3 1.1 44496 11640 ? SN 16:07 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27454 0.2 1.2 44844 13028 ? SN 16:07 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27455 0.6 1.3 44836 13776 ? SN 16:07 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27460 1.0 1.3 45092 13924 ? SN 16:07 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27468 1.1 1.6 46888 16928 ? SN 16:07 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27470 0.3 1.2 44888 12844 ? SN 16:07 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27471 0.4 1.3 44720 13788 ? SN 16:07 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27472 0.9 1.4 45548 14364 ? SN 16:07 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27473 0.8 1.4 44860 14620 ? RN 16:07 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27490 0.2 1.1 44168 11536 ? SN 16:07 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27492 0.4 1.4 44668 15148 ? SN 16:08 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27493 0.7 1.3 45764 13976 ? SN 16:08 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27611 0.9 1.4 46736 15004 ? SN 16:08 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27612 0.4 1.4 44952 14312 ? SN 16:08 0:00 /usr/local/apache/bin/httpd -DSSL nobody 27613 0.8 1.3 46744 13644 ? SN 16:08 0:00 /usr/local/apache/bin/httpd -DSSL nobody 29636 1.0 1.3 44976 13828 ? SN 16:08 0:00 /usr/local/apache/bin/httpd -DSSL nobody 29640 0.0 0.7 43588 7412 ? SN 16:08 0:00 /usr/local/apache/bin/httpd -DSSL nobody 29641 1.5 1.3 45128 13960 ? SN 16:08 0:00 /usr/local/apache/bin/httpd -DSSL nobody 29645 0.4 1.1 44436 11520 ? SN 16:08 0:00 /usr/local/apache/bin/httpd -DSSL mailnull 29648 0.0 0.1 6672 1972 ? S 16:08 0:00 /usr/sbin/exim -bd -q60m nobody 29677 0.6 1.0 44540 11176 ? SN 16:08 0:00 /usr/local/apache/bin/httpd -DSSL mailnull 29737 0.0 0.0 0 0 ? Z 16:09 0:00 [exim <defunct>] nobody 29738 0.0 0.2 4576 2508 ? SNL 16:09 0:00 proftpd: connected: 127.0.0.1 (127.0.0.1:19603) nobody 29747 0.0 0.7 43568 7412 ? SN 16:09 0:00 /usr/local/apache/bin/httpd -DSSL mailnull 29749 0.5 0.3 7488 3808 ? S 16:09 0:00 /usr/sbin/exim -bd -q60m nobody 29750 0.0 0.7 43588 7240 ? SN 16:09 0:00 /usr/local/apache/bin/httpd -DSSL nobody 29751 0.0 0.7 43572 7196 ? SN 16:09 0:00 /usr/local/apache/bin/httpd -DSSL root 29755 0.0 0.0 2924 828 pts/0 R 16:09 0:00 ps -aux
Thanks a lot!
