Andreas Larsson Andreas Larsson - 1 month ago 10
Apache Configuration Question

Problems with Apache servers and A LOT of httpd processes

We have a problem with our Apache server. It's really slow and cpu usage is really high (sometimes showing a constant 100%).

It's on a ubuntu system.

Any ideas what might be causing this?

"top" shows this (there's probably even more httpd processes that didn't fit in the terminal window):

top - 11:33:19 up 1:07, 2 users, load average: 37.79, 35.90, 29.14
Tasks: 86 total, 10 running, 76 sleeping, 0 stopped, 0 zombieCpu(s): 90.6%us, 9.4%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 786432k total, 379584k used, 406848k free, 0k buffersSwap: 0k total, 0k used, 0k free, 0k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5507 nobody 16 0 54540 19m 5204 S 4 2.6 0:03.70 httpd 5653 nobody 17 0 59996 24m 4808 R 4 3.2 0:01.98 httpd
3277 nobody 15 0 201m 51m 6724 R 2 6.7 2:14.30 mysqld 5260 nobody 15 0 53740 18m 4972 S 2 2.4 0:06.53 httpd
5333 nobody 15 0 54536 19m 5152 S 2 2.5 0:06.50 httpd 3923 nobody 15 0 52644 18m 6372 S 1 2.4 0:16.80 httpd
5519 nobody 15 0 51252 15m 4792 S 1 2.0 0:03.02 httpd 4048 nobody 15 0 52808 17m 5040 R 1 2.3 0:13.86 httpd
5240 nobody 15 0 53980 18m 4916 S 1 2.4 0:08.42 httpd
5249 nobody 15 0 52908 17m 5188 S 1 2.3 0:08.23 httpd 5385 nobody 15 0 52568 16m 4824 S 1 2.2 0:04.19 httpd
5395 nobody 16 0 52536 16m 4804 S 1 2.1 0:03.90 httpd 5422 nobody 16 0 51408 15m 4888 S 1 2.1 0:03.49 httpd
5503 nobody 15 0 52536 16m 4800 S 1 2.1 0:02.67 httpd 3646 nobody 15 0 52572 16m 5012 S 1 2.2 0:25.42 httpd
3948 nobody 15 0 52844 17m 5008 R 1 2.3 0:17.15 httpd 5283 nobody 15 0 54000 18m 4944 S 1 2.4 0:07.05 httpd
5300 nobody 15 0 54748 19m 5276 S 1 2.6 0:06.20 httpd 5304 nobody 15 0 52876 17m 4904 S 1 2.2 0:06.45 httpd
5305 nobody 15 0 52812 17m 4884 S 1 2.2 0:06.02 httpd 5400 nobody 16 0 52060 16m 4800 R 1 2.1 0:04.10 httpd
5423 nobody 15 0 54764 19m 5196 S 1 2.6 0:04.37 httpd 5433 nobody 15 0 52168 16m 4844 R 1 2.1 0:03.38 httpd
5497 nobody 15 0 52572 16m 4820 S 1 2.2 0:02.83 httpd 5498 nobody 15 0 53076 17m 4864 S 1 2.3 0:03.14 httpd
5502 nobody 15 0 51260 15m 4804 S 1 2.0 0:02.70 httpd 5505 nobody 15 0 54920 19m 4976 S 1 2.5 0:02.68 httpd
5546 nobody 15 0 52584 16m 4664 S 1 2.2 0:02.48 httpd 5550 nobody 15 0 51244 15m 4796 S 1 2.0 0:02.42 httpd
5652 nobody 15 0 51976 17m 5912 S 1 2.3 0:01.24 httpd 5659 nobody 15 0 55012 19m 4980 S 1 2.6 0:01.25 httpd
5707 nobody 15 0 51416 15m 4880 S 1 2.1 0:00.09 httpd 5710 nobody 15 0 51240 15m 4792 S 1 2.0 0:00.14 httpd
5399 nobody 15 0 52568 16m 4856 S 1 2.2 0:03.86 httpd 5435 nobody 15 0 52596 16m 4864 S 1 2.2 0:02.96 httpd
5467 nobody 15 0 52316 16m 4856 R 1 2.1 0:02.93 httpd 5501 nobody 15 0 52956 17m 4844 S 1 2.2 0:02.79 httpd
5525 nobody 15 0 54508 19m 4916 S 1 2.5 0:03.41 httpd 5547 nobody 15 0 52604 18m 5708 S 1 2.3 0:02.77 httpd
5590 root 15 0 19208 1360 1044 R 1 0.2 0:00.39 top 5627 nobody 15 0 55648 20m 5164 S 1 2.6 0:02.56 httpd
5629 nobody 15 0 51248 15m 4792 S 1 2.0 0:01.62 httpd 5647 nobody 15 0 52956 17m 4884 S 1 2.2 0:01.16 httpd
5656 nobody 15 0 52588 17m 4884 S 1 2.2 0:00.94 httpd 5665 nobody 15 0 51244 15m 4792 S 1 2.0 0:01.36 httpd
5709 nobody 15 0 51928 15m 4628 S 1 2.1 0:00.02 httpd 5711 nobody 16 0 51148 15m 4600 S 1 2.0 0:00.02 httpd

Answer

Apache needs to fork a process to handle each connected client. Read more about that here.

The upshot is that you have lots of processes because you have lots of connections. If your server is running slowly as a result then here are a few options:

  1. Optimise your application: profile your application's code and try and optimise the areas that place the most load on the server.
  2. Consider changing to a lower impact web server, such as nginx. This could increase the number of connections that your server can handle.
  3. Add page caching, using something like varnish
  4. Add another server, and load balance between them
  5. Increase the capabilities of the existing server (e.g. increase RAM, upgrade processor, install a faster hard disk).

All of these options have a cost, in time or money. Unfortunately that's just the way it is :)

The quickest wins in my opinion would probably be options 2 and 3. But it means downtime while they're being installed, or having a backup server to work on.