Centos Server Recently slowed down significantly

Posted on

Centos Server Recently slowed down significantly – Managing your servers can streamline the performance of your team by allowing them to complete complex tasks faster. Plus, it can enable them to detect problems early on before they get out of hand and compromise your business. As a result, the risk of experiencing operational setbacks is drastically lower.

But the only way to make the most of your server management is to perform it correctly. And to help you do so, this article will share nine tips on improving your server management and fix some problem about linux, centos, performance-tuning, , .

I have Dedicated Server on 1and1 with

RAM - 16 GB
HD 1000GB
CPU: Intel(R) Xeon(R) CPU E3-1270 v3 @ 3.50GHz, 4 cores
Centos 6
Apache/2.2.15
PHP 5.3.3
Mysql 

I have a video sharing website (on joomla) with about 5000 media items
3500 media items on Amazon and rest is on server (about 489GB) and recently website slowed down significantly. THe pages started to load for 5 – 10 seconds, backend about 10 – 20 seconds. 2 – 3 days before it worked much better, not perfect but much better.

I need to identify the problem, myabe you can suggest some commands for tracking performance wich would help me to find the problem?? Because something really went wrong with server.

———-‘top’ result————–

top - 10:35:45 up 17:57,  1 user,  load average: 41.21, 46.46, 55.28
Tasks: 327 total,   7 running, 320 sleeping,   0 stopped,   0 zombie
Cpu(s): 62.6%us,  0.0%sy,  0.0%ni, 28.8%id,  8.6%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  16294080k total, 15210716k used,  1083364k free,    55924k buffers
Swap:  4194288k total,    30268k used,  4164020k free, 11651236k cached


1457 apache    20   0 1167m 157m 4632 S 34.2  1.0  16:17.50 ffmpeg
614 root      20   0  265m  56m 4260 R 33.2  0.4  20:00.14 ffmpeg
1059 root      20   0  265m  55m 4260 R 33.2  0.3  16:52.33 ffmpeg
2733 apache    20   0  266m  55m 4256 R 33.2  0.4   5:25.97 ffmpeg
2786 apache    20   0  266m  56m 4264 R 33.2  0.4   4:50.05 ffmpeg
2854 root      20   0  147m  25m 3580 R 33.2  0.2   3:25.02 ffmpeg
29788 root      20   0  267m  56m 4268 R 33.2  0.4  34:31.32 ffmpeg
31612 root      20   0  266m  55m 4264 R 33.2  0.4  27:22.21 ffmpeg
32589 root      20   0  266m  57m 4264 R 33.2  0.4  24:25.37 ffmpeg
2625 apache    20   0  265m  54m 4256 R 32.9  0.3   6:18.87 ffmpeg
2895 apache    20   0  266m  57m 4260 R 32.9  0.4   2:22.71 ffmpeg
28797 root      20   0  267m  58m 4268 R 32.9  0.4  46:03.94 ffmpeg
32297 root      20   0  266m  57m 4264 R 32.9  0.4  25:24.76 ffmpeg
28343 root      20   0  268m  57m 4268 R 32.6  0.4  54:09.69 ffmpeg
1436 root      20   0  265m  55m 4260 R 32.2  0.4  15:02.93 ffmpeg
2138 root      20   0  265m  55m 4260 R 32.2  0.3  10:41.26 ffmpeg
30435 root      20   0  266m  56m 4264 R 32.2  0.4  30:41.72 ffmpeg
2647 apache    20   0  266m  57m 4272 R 31.9  0.4   6:15.76 ffmpeg
2978 root      20   0  144m  22m 3572 R 31.9  0.1   1:15.28 ffmpeg

[root@u17669867 ~]# iostat
Linux 2.6.32-431.20.5.el6.x86_64 (u17669867)    08/15/2014      _x86_64_        (8 CPU)

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          78.37   15.62    0.38    2.05    0.00    3.58

Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sdc               8.57       495.09      2368.59   33674022  161101560
sda               8.41        30.21       171.06    2054642   11634608
sdd               6.85       390.15      2366.70   26536668  160972704
sdb               4.84         2.84       135.32     193088    9203664
md3               4.35        20.51        27.92    1395238    1898808
md1               5.24         2.25        41.40     153236    2816112
md11            303.70       885.20      2368.26   60207426  161079040
dm-0            303.66       885.19      2368.26   60206658  161079048
dm-1              0.12         4.33         0.01     294802        720
dm-2              3.70        16.15        27.91    1098146    1898064
dm-3              0.00         0.02         0.00       1602         24

Solution :

Start by tracing what is taking too long with Chrome/IE developer tools (network tab).

Confirming that the slowness is happening for all requests irrespective of type, trace the path from end user to your server (traceroute) looking for any excessive delays. Test from multiple places.

On the server, check the number of open connections (netstat -antp | grep httpd), you may be hitting a limit of concurrent users and the connections/requests get queued up waiting for a slot.

Also check CPU (top, ps aux), memory (free -m) and disk usage (iostat).

EDIT: Your top results show CPU starvation and the iostat results aren’t any better. Consider adding another server or move the CPU-intensive work away from the web server.

According to top, it looks like something is using up a good amount of CPU. Update your post with the process list according to top. More than likely, something is pulling a lot of I/O on the server and exhausting RAM as most of your swap is cached (so, recently used). The load should not be that high on a healthy server.

Leave a Reply

Your email address will not be published. Required fields are marked *