Server Lag Spikes
I dont really know whats causing it, any help will be greatly appreciated.
Here is the server operating normally:
Code:
09:16:33 up 203 days, 22:04, 1 user, load average: 0.40, 0.65, 0.73
105 processes: 104 sleeping, 1 running, 0 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 0.7% 0.0% 0.7% 0.0% 0.0% 0.0% 98.4%
Mem: 1022532k av, 997740k used, 24792k free, 0k shrd, 73316k buff
758576k actv, 142580k in_d, 17356k in_c
Swap: 2040244k av, 54896k used, 1985348k free 760468k cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
12722 maddev 21 0 1252 1252 896 R 1.5 0.1 0:00 0 top
1 root 15 0 508 476 448 S 0.0 0.0 0:25 0 init
2 root 15 0 0 0 0 SW 0.0 0.0 0:04 0 keventd
3 root 15 0 0 0 0 SW 0.0 0.0 0:47 0 kapmd
4 root 34 19 0 0 0 SWN 0.0 0.0 0:00 0 ksoftirqd/0
7 root 15 0 0 0 0 SW 0.0 0.0 0:06 0 bdflush
5 root 15 0 0 0 0 SW 0.0 0.0 8:50 0 kswapd
6 root 15 0 0 0 0 SW 0.0 0.0 15:07 0 kscand
8 root 15 0 0 0 0 SW 0.0 0.0 0:12 0 kupdated
9 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 mdrecoveryd
13 root 15 0 0 0 0 SW 0.0 0.0 30:04 0 kjournald
68 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 khubd
2146 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 kjournald
2506 root 15 0 580 548 496 S 0.0 0.0 1:05 0 syslogd
2510 root 15 0 440 384 380 S 0.0 0.0 0:00 0 klogd
2536 rpc 15 0 540 460 456 S 0.0 0.0 0:00 0 portmap
2548 root RT 0 536 436 408 S 0.0 0.0 0:00 0 auditd
2599 root 24 0 480 428 424 S 0.0 0.0 0:00 0 apmd
2676 root 15 0 808 644 560 S 0.0 0.0 0:32 0 sshd
2690 root 24 0 548 432 428 S 0.0 0.0 0:00 0 xinetd
2718 smmsp 15 0 1088 628 536 S 0.0 0.0 0:01 0 sendmail
2732 root 15 0 392 372 316 S 0.0 0.0 0:01 0 crond
2780 xfs 15 0 2140 292 264 S 0.0 0.0 0:00 0 xfs
2789 daemon 15 0 348 328 284 S 0.0 0.0 0:00 0 atd
2802 root 15 0 284 264 224 S 0.0 0.0 0:01 0 mdadm
2813 root 15 0 12152 11M 1372 S 0.0 1.1 0:01 0 mdmpd
2832 root 15 0 3992 1356 956 S 0.0 0.1 0:09 0 miniserv.pl
2836 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2837 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2838 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2839 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2840 root 22 0 124 72 68 S 0.0 0.0 0:00 0 mingetty
2841 root 22 0 292 240 236 S 0.0 0.0 0:00 0 mingetty
17828 maddev 15 0 600 252 248 S 0.0 0.0 0:00 0 screen
17829 maddev 25 0 824 604 600 S 0.0 0.0 0:00 0 bash
Code:
0:52:53 up 177 days, 13:41, 1 user, load average: 32.19, 19.66, 10.23
6 root 16 0 0 0 0 SW 3.4 0.0 10:45 0 kscand
18172 maddev 15 0 1396 1396 896 R 3.4 0.1 0:00 0 top
17821 mysql 15 0 36976 24M 1368 S 1.7 2.4 0:00 0 mysqld
1 root 15 0 508 472 448 S 0.0 0.0 0:22 0 init
2 root 15 0 0 0 0 SW 0.0 0.0 0:03 0 keventd
3 root 15 0 0 0 0 SW 0.0 0.0 0:41 0 kapmd
4 root 34 19 0 0 0 SWN 0.0 0.0 0:00 0 ksoftirqd/0
7 root 15 0 0 0 0 SW 0.0 0.0 0:05 0 bdflush
5 root 15 0 0 0 0 SW 0.0 0.0 6:51 0 kswapd
8 root 15 0 0 0 0 SW 0.0 0.0 0:10 0 kupdated
9 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 mdrecoveryd
13 root 15 0 0 0 0 DW 0.0 0.0 28:39 0 kjournald
68 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 khubd
2146 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 kjournald
2506 root 15 0 588 552 504 S 0.0 0.0 0:57 0 syslogd
2510 root 15 0 444 388 384 S 0.0 0.0 0:00 0 klogd
2536 rpc 15 0 560 480 476 S 0.0 0.0 0:00 0 portmap
2548 root RT 0 536 416 408 S 0.0 0.0 0:00 0 auditd
2599 root 24 0 480 428 424 S 0.0 0.0 0:00 0 apmd
2676 root 15 0 796 592 548 S 0.0 0.0 0:26 0 sshd
2690 root 24 0 548 432 428 S 0.0 0.0 0:00 0 xinetd
2718 smmsp 15 0 1104 556 552 S 0.0 0.0 0:01 0 sendmail
2732 root 15 0 424 400 348 S 0.0 0.0 0:01 0 crond
2780 xfs 15 0 2132 260 256 S 0.0 0.0 0:00 0 xfs
2789 daemon 15 0 356 312 292 S 0.0 0.0 0:00 0 atd
2802 root 15 0 292 236 232 S 0.0 0.0 0:00 0 mdadm
2813 root 15 0 12152 11M 1372 S 0.0 1.1 0:01 0 mdmpd
2832 root 15 0 3992 1300 956 S 0.0 0.1 0:08 0 miniserv.pl
2836 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2837 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2838 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2839 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2840 root 22 0 124 72 68 S 0.0 0.0 0:00 0 mingetty
2841 root 22 0 308 256 252 S 0.0 0.0 0:00 0 mingetty
17828 maddev 15 0 608 260 256 S 0.0 0.0 0:00 0 screen
Code:
21:59:00 up 182 days, 10:47, 1 user, load average: 40.48, 19.47, 8.39
70 processes: 68 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 0.0% 0.0% 0.0% 0.0% 0.0% 0.0% 100.0%
Mem: 1022532k av, 655076k used, 367456k free, 0k shrd, 17608k buff
469484k actv, 109716k in_d, 12120k in_c
Swap: 2040244k av, 28212k used, 2012032k free 586528k cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
1 root 15 0 508 476 448 S 0.0 0.0 0:23 0 init
2 root 15 0 0 0 0 SW 0.0 0.0 0:04 0 keventd
3 root 15 0 0 0 0 SW 0.0 0.0 0:43 0 kapmd
4 root 34 19 0 0 0 SWN 0.0 0.0 0:00 0 ksoftirqd/0
7 root 15 0 0 0 0 SW 0.0 0.0 0:06 0 bdflush
5 root 15 0 0 0 0 SW 0.0 0.0 7:14 0 kswapd
6 root 15 0 0 0 0 SW 0.0 0.0 11:34 0 kscand
8 root 15 0 0 0 0 SW 0.0 0.0 0:11 0 kupdated
9 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 mdrecoveryd
13 root 15 0 0 0 0 SW 0.0 0.0 28:55 0 kjournald
68 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 khubd
2146 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 kjournald
2506 root 15 0 588 552 504 S 0.0 0.0 0:59 0 syslogd
2510 root 15 0 444 388 384 S 0.0 0.0 0:00 0 klogd
2536 rpc 15 0 560 480 476 S 0.0 0.0 0:00 0 portmap
2548 root RT 0 528 424 400 S 0.0 0.0 0:00 0 auditd
2599 root 24 0 480 428 424 S 0.0 0.0 0:00 0 apmd
2676 root 15 0 804 600 556 S 0.0 0.0 0:27 0 sshd
2690 root 24 0 548 432 428 S 0.0 0.0 0:00 0 xinetd
2718 smmsp 15 0 1112 564 560 S 0.0 0.0 0:01 0 sendmail
2732 root 15 0 416 388 340 S 0.0 0.0 0:01 0 crond
2780 xfs 15 0 2144 292 268 S 0.0 0.0 0:00 0 xfs
2789 daemon 15 0 352 328 288 S 0.0 0.0 0:00 0 atd
2802 root 15 0 288 268 228 S 0.0 0.0 0:00 0 mdadm
2813 root 15 0 12152 11M 1372 S 0.0 1.1 0:01 0 mdmpd
2832 root 15 0 3992 1276 956 S 0.0 0.1 0:08 0 miniserv.pl
2836 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2837 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2838 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2839 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2840 root 22 0 124 72 68 S 0.0 0.0 0:00 0 mingetty
2841 root 22 0 300 248 244 S 0.0 0.0 0:00 0 mingetty
17828 maddev 15 0 608 260 256 S 0.0 0.0 0:00 0 screen
Code:
10:25:39 up 189 days, 23:13, 1 user, load average: 29.73, 12.17, 6.57
284 processes: 280 sleeping, 4 running, 0 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 14.9% 0.0% 1.1% 0.3% 0.0% 83.5% 0.0%
Mem: 1022532k av, 1012256k used, 10276k free, 0k shrd, 8700k buff
779880k actv, 146132k in_d, 13100k in_c
Swap: 2040244k av, 282056k used, 1758188k free 580432k cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
13989 apache 15 0 11460 9084 4220 D 6.1 0.8 0:00 0 httpd
14032 apache 16 0 11420 8932 4220 R 2.4 0.8 0:00 0 httpd
13942 apache 15 0 10204 7792 4208 D 0.7 0.7 0:00 0 httpd
14192 apache 15 0 9692 7952 4168 S 0.5 0.7 0:00 0 httpd
14193 apache 15 0 9692 7712 4164 D 0.5 0.7 0:00 0 httpd
14363 maddev 15 0 1380 1376 848 R 0.5 0.1 0:00 0 top
13883 apache 15 0 11540 9204 4256 D 0.3 0.9 0:01 0 httpd
14197 apache 15 0 10504 7612 4188 S 0.3 0.7 0:00 0 httpd
14198 apache 15 0 9724 7144 4156 S 0.3 0.6 0:00 0 httpd
5 root 15 0 0 0 0 SW 0.1 0.0 7:49 0 kswapd
13691 apache 15 0 11912 8564 4336 D 0.1 0.8 0:04 0 httpd
13868 apache 15 0 10928 8836 4256 D 0.1 0.8 0:00 0 httpd
13878 apache 15 0 11476 9108 4216 S 0.1 0.8 0:01 0 httpd
13885 apache 15 0 11488 9128 4220 S 0.1 0.8 0:01 0 httpd
13901 apache 15 0 12012 9M 4252 D 0.1 1.0 0:01 0 httpd
13906 apache 15 0 9928 8092 4224 D 0.1 0.7 0:00 0 httpd
13931 apache 15 0 10476 8176 4232 D 0.1 0.7 0:00 0 httpd
13943 apache 15 0 10860 8920 4284 D 0.1 0.8 0:00 0 httpd
13970 mysql 15 0 38672 24M 1300 S 0.1 2.4 0:00 0 mysqld
13974 apache 15 0 10228 7996 4192 D 0.1 0.7 0:00 0 httpd
13982 apache 15 0 10928 8544 4204 D 0.1 0.8 0:00 0 httpd
13995 apache 15 0 10860 8380 4224 D 0.1 0.8 0:00 0 httpd
14061 apache 15 0 9572 7932 4176 D 0.1 0.7 0:00 0 httpd
14077 apache 15 0 9252 7612 4168 S 0.1 0.7 0:00 0 httpd
14099 apache 15 0 9260 7500 4176 S 0.1 0.7 0:00 0 httpd
14194 mysql 15 0 38672 24M 1300 S 0.1 2.4 0:00 0 mysqld
14195 apache 15 0 10716 8896 4216 D 0.1 0.8 0:00 0 httpd
14268 mysql 15 0 38672 24M 1300 S 0.1 2.4 0:00 0 mysqld
14401 root 20 0 6204 4124 3620 R 0.1 0.4 0:00 0 httpd
1 root 15 0 508 472 448 S 0.0 0.0 0:24 0 init
2 root 15 0 0 0 0 SW 0.0 0.0 0:04 0 keventd
3 root 15 0 0 0 0 SW 0.0 0.0 0:44 0 kapmd
4 root 34 19 0 0 0 SWN 0.0 0.0 0:00 0 ksoftirqd/0
7 root 15 0 0 0 0 SW 0.0 0.0 0:06 0 bdflush
6 root 15 0 0 0 0 SW 0.0 0.0 12:57 0 kscand
Code:
04:43:53 up 200 days, 17:32, 1 user, load average: 23.25, 9.69, 4.33
277 processes: 274 sleeping, 3 running, 0 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 14.5% 0.0% 2.3% 0.7% 0.0% 82.3% 0.0%
Mem: 1022532k av, 618448k used, 404084k free, 0k shrd, 6260k buff
533328k actv, 38064k in_d, 2700k in_c
Swap: 2040244k av, 216880k used, 1823364k free 223892k cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
1447 mysql 15 0 35988 22M 1364 S 7.1 2.2 0:00 0 mysqld
1484 apache 15 0 9344 7288 4200 S 1.3 0.7 0:00 0 httpd
1207 apache 15 0 12396 9.8M 4440 S 0.7 0.9 0:02 0 httpd
1568 maddev 15 0 1424 1424 904 R 0.7 0.1 0:00 0 top
1377 apache 15 0 10568 7260 4204 D 0.5 0.7 0:00 0 httpd
1418 apache 15 0 10216 7348 4204 D 0.5 0.7 0:00 0 httpd
1422 apache 15 0 11560 8988 4260 S 0.5 0.8 0:00 0 httpd
1428 apache 15 0 11504 8772 4248 S 0.5 0.8 0:00 0 httpd
1461 apache 15 0 9916 6572 4192 S 0.5 0.6 0:00 0 httpd
1465 apache 15 0 9916 6608 4192 S 0.5 0.6 0:00 0 httpd
1474 apache 15 0 9916 7076 4192 S 0.5 0.6 0:00 0 httpd
1215 apache 15 0 12160 8156 4288 D 0.2 0.7 0:01 0 httpd
1216 apache 15 0 11128 7136 4300 D 0.2 0.6 0:01 0 httpd
1368 apache 15 0 10392 8332 4280 D 0.2 0.8 0:00 0 httpd
1371 mysql 15 0 35988 22M 1364 S 0.2 2.2 0:00 0 mysqld
1396 apache 15 0 10532 7556 4204 D 0.2 0.7 0:00 0 httpd
1427 apache 15 0 10204 7408 4244 D 0.2 0.7 0:00 0 httpd
1444 apache 15 0 9916 6352 4192 S 0.2 0.6 0:00 0 httpd
1469 apache 15 0 10664 7704 4208 D 0.2 0.7 0:00 0 httpd
1470 mysql 15 0 35988 22M 1364 S 0.2 2.2 0:00 0 mysqld
1473 apache 15 0 9916 7040 4192 S 0.2 0.6 0:00 0 httpd
1489 apache 15 0 8480 6880 4276 S 0.2 0.6 0:00 0 httpd
1495 apache 15 0 10924 9236 4216 S 0.2 0.9 0:00 0 httpd
1608 mysql 15 0 35988 22M 1364 S 0.2 2.2 0:00 0 mysqld
1 root 15 0 508 472 448 S 0.0 0.0 0:24 0 init
2 root 15 0 0 0 0 SW 0.0 0.0 0:04 0 keventd
3 root 15 0 0 0 0 SW 0.0 0.0 0:47 0 kapmd
4 root 34 19 0 0 0 SWN 0.0 0.0 0:00 0 ksoftirqd/0
7 root 15 0 0 0 0 SW 0.0 0.0 0:06 0 bdflush
5 root 15 0 0 0 0 SW 0.0 0.0 8:38 0 kswapd
6 root 15 0 0 0 0 SW 0.0 0.0 14:48 0 kscand
8 root 15 0 0 0 0 SW 0.0 0.0 0:11 0 kupdated
9 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 mdrecoveryd
13 root 15 0 0 0 0 SW 0.0 0.0 29:54 0 kjournald
68 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 khubd
Code:
22:01:41 up 196 days, 10:49, 1 user, load average: 1.87, 1.76, 1.74
115 processes: 113 sleeping, 2 running, 0 zombie, 0 stopped
CPU states: cpu user nice system irq softirq iowait idle
total 6.1% 0.0% 0.1% 0.0% 0.0% 0.0% 93.6%
Mem: 1022532k av, 998516k used, 24016k free, 0k shrd, 17560k buff
775432k actv, 141536k in_d, 14668k in_c
Swap: 2040244k av, 48904k used, 199****k free 809792k cached
PID USER PRI NI SIZE RSS SHARE STAT %CPU %MEM TIME CPU COMMAND
17697 apache 16 0 11600 9.9M 4472 S 2.7 0.9 0:05 0 httpd
18113 apache 16 0 10264 8716 4292 S 2.1 0.8 0:00 0 httpd
17812 apache 15 0 12320 10M 4344 S 0.3 1.0 0:05 0 httpd
17911 apache 15 0 11288 9808 4428 S 0.3 0.9 0:01 0 httpd
18193 maddev 17 0 1268 1268 904 R 0.1 0.1 0:00 0 top
1 root 15 0 508 476 448 S 0.0 0.0 0:24 0 init
2 root 15 0 0 0 0 SW 0.0 0.0 0:04 0 keventd
3 root 15 0 0 0 0 SW 0.0 0.0 0:46 0 kapmd
4 root 34 19 0 0 0 SWN 0.0 0.0 0:00 0 ksoftirqd/0
7 root 15 0 0 0 0 SW 0.0 0.0 0:06 0 bdflush
5 root 15 0 0 0 0 SW 0.0 0.0 8:21 0 kswapd
6 root 15 0 0 0 0 SW 0.0 0.0 14:04 0 kscand
8 root 15 0 0 0 0 SW 0.0 0.0 0:11 0 kupdated
9 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 mdrecoveryd
13 root 15 0 0 0 0 SW 0.0 0.0 29:41 0 kjournald
68 root 25 0 0 0 0 SW 0.0 0.0 0:00 0 khubd
2146 root 15 0 0 0 0 SW 0.0 0.0 0:00 0 kjournald
2506 root 15 0 588 552 504 S 0.0 0.0 1:03 0 syslogd
2510 root 15 0 452 396 392 S 0.0 0.0 0:00 0 klogd
2536 rpc 15 0 560 480 476 S 0.0 0.0 0:00 0 portmap
2548 root RT 0 536 436 408 S 0.0 0.0 0:00 0 auditd
2599 root 24 0 480 428 424 S 0.0 0.0 0:00 0 apmd
2676 root 15 0 804 636 556 S 0.0 0.0 0:30 0 sshd
2690 root 24 0 548 432 428 S 0.0 0.0 0:00 0 xinetd
2718 smmsp 15 0 1084 536 532 S 0.0 0.0 0:01 0 sendmail
2732 root 15 0 408 388 332 S 0.0 0.0 0:01 0 crond
2780 xfs 15 0 2140 268 264 S 0.0 0.0 0:00 0 xfs
2789 daemon 15 0 348 328 284 S 0.0 0.0 0:00 0 atd
2802 root 15 0 284 264 224 S 0.0 0.0 0:01 0 mdadm
2813 root 15 0 12152 11M 1372 S 0.0 1.1 0:01 0 mdmpd
2832 root 15 0 3992 1304 956 S 0.0 0.1 0:09 0 miniserv.pl
2836 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2837 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2838 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
2839 root 22 0 120 72 68 S 0.0 0.0 0:00 0 mingetty
