多并发连接+ nginx的php5-fpm设置

Question

请帮我调整一下php5-fpm和nginx的配置。

问题是我的php5-fpm日志一直在报告缓慢的脚本并杀死一个子线程。

专用服务器，四xeon，32 Ram内存。1 php应用程序/站点运行。

Php应用程序:简单，搜索引擎，结果触发卷曲请求。页面加载时间通常为每次搜索2-3秒.

以下是我认为正在发生的事情：

我得到了750个同时执行搜索的php用户。由于内存限制，我只能设置pm.max_children = 400。我假设每个用户(子线程)为50 so，以便=20 so。我假设每个用户=一个子线程。因此，pm.max_children不足以覆盖使用3秒搜索的750个活动php用户。

所以我想我看到的是用户排队，因为我看到3秒变成4-7秒。当用户排队时，我认为脚本正在缓慢触发错误日志消息，php5-fpm会杀死子程序吗？

这就是我认为正在发生的事情。我已经提供了我的错误日志输出，nginx，php5-fpm配置如下。

如果我可以调整我的配置，如果pm.max_children至少应该等于您最大的并发用户，请记住我的php搜索打开了大约3秒，我将非常感激您的建议。我需要更多的内存还是更多的服务器？

这是我的记忆，但我大约30分钟前才重新启动nginx。

:/var/log# free -m
             total       used       free     shared    buffers     cached
Mem:         32151      26175       5975          0        186      13334
-/+ buffers/cache:      12654      19496
Swap:        32739          5      32734

php5-fpm: www.conf:流程管理器设置为静态

我使用静态，因为我认为所有的孩子都可以立即使用，而不是产卵时间，而且我只在盒子上运行一个应用程序。

;pm = dynamic
pm = static

;pm.max_children = 10
pm.max_children = 400


;pm.start_servers = 4
pm.start_servers = 150


;pm.min_spare_servers = 2
pm.min_spare_servers = 32


;pm.max_spare_servers = 6
pm.max_spare_servers = 64


;pm.max_requests = 500
pm.max_requests = 10000

php5-fpm日志中的错误

我要澄清的是，我在高负荷下看到的行为是，750名用户同时缓存和不缓存的搜索结果开始花费更长的时间。即缓存>1秒，非缓存在4到7秒之间。因此，当用户排队等待时，我认为搜索时间会增加，而且还会上升到脚本在负载下运行缓慢的程度。触发通知，孩子就会被杀。

这是在重启之后

[04-Jun-2013 20:11:07] NOTICE: Finishing ...
[04-Jun-2013 20:11:11] NOTICE: exiting, bye-bye!
[04-Jun-2013 20:11:12] NOTICE: fpm is running, pid 17899
[04-Jun-2013 20:11:12] NOTICE: ready to handle connections
[04-Jun-2013 20:27:28] WARNING: [pool www] child 18200, script '/home/site/public_html/index.php' (request: "POST /index.php") executing too slow (10.827363 sec), logging
[04-Jun-2013 20:27:28] WARNING: [pool www] child 18138, script '/home/site/public_html/index.php' (request: "POST /index.php") executing too slow (10.827034 sec), logging
[04-Jun-2013 20:27:28] NOTICE: child 18138 stopped for tracing
[04-Jun-2013 20:27:28] NOTICE: about to trace 18138
[04-Jun-2013 20:27:28] NOTICE: finished trace of 18138
[04-Jun-2013 20:27:28] NOTICE: child 18200 stopped for tracing
[04-Jun-2013 20:27:28] NOTICE: about to trace 18200
[04-Jun-2013 20:27:28] NOTICE: finished trace of 18200
[04-Jun-2013 20:52:52] WARNING: [pool www] child 17948, script '/home/site/public_html/index.php' (request: "GET /index.php") executing too slow (11.724081 sec), logging
[04-Jun-2013 20:52:52] NOTICE: child 17948 stopped for tracing
[04-Jun-2013 20:52:52] NOTICE: about to trace 17948
[04-Jun-2013 20:52:52] ERROR: failed to ptrace(PEEKDATA) pid 17948: Input/output error (5)
[04-Jun-2013 20:52:52] NOTICE: finished trace of 17948
[04-Jun-2013 20:58:22] WARNING: [pool www] child 18287, script '/home/site/public_html/index.php' (request: "POST /index.php") executing too slow (10.701504 sec), logging
[04-Jun-2013 20:58:22] NOTICE: child 18287 stopped for tracing
[04-Jun-2013 20:58:22] NOTICE: about to trace 18287
[04-Jun-2013 20:58:22] NOTICE: finished trace of 18287
[04-Jun-2013 21:19:22] WARNING: [pool www] child 18224, script '/home/site/public_html/index.php' (request: "GET /index.php") executing too slow (10.005466 sec), logging
[04-Jun-2013 21:19:22] WARNING: [pool www] child 18197, script '/home/site/public_html/index.php' (request: "GET /index.php") executing too slow (12.141221 sec), logging
[04-Jun-2013 21:19:22] WARNING: [pool www] child 17946, script '/home/site/public_html/index.php' (request: "GET /index.php") executing too slow (11.107080 sec), logging
[04-Jun-2013 21:19:22] NOTICE: child 17946 stopped for tracing
[04-Jun-2013 21:19:22] NOTICE: about to trace 17946
[04-Jun-2013 21:19:22] NOTICE: finished trace of 17946
[04-Jun-2013 21:19:22] NOTICE: child 18197 stopped for tracing
[04-Jun-2013 21:19:22] NOTICE: about to trace 18197
[04-Jun-2013 21:19:22] NOTICE: finished trace of 18197
[04-Jun-2013 21:19:22] NOTICE: child 18224 stopped for tracing
[04-Jun-2013 21:19:22] NOTICE: about to trace 18224
[04-Jun-2013 21:19:22] NOTICE: finished trace of 18224
[04-Jun-2013 21:19:26] WARNING: [pool www] child 18197, script '/home/site/public_html/index.php' (request: "GET /index.php") execution timed out (15.475021 sec), terminating
[04-Jun-2013 21:19:26] WARNING: [pool www] child 18055, script '/home/site/public_html/index.php' (request: "GET /index.php") executing too slow (12.927407 sec), logging
[04-Jun-2013 21:19:26] NOTICE: child 18055 stopped for tracing
[04-Jun-2013 21:19:26] NOTICE: about to trace 18055
[04-Jun-2013 21:19:26] NOTICE: finished trace of 18055
[04-Jun-2013 21:19:26] WARNING: [pool www] child 18197 exited on signal 15 (SIGTERM) after 4094.193190 seconds from start
[04-Jun-2013 21:19:26] NOTICE: [pool www] child 5137 started
[04-Jun-2013 21:24:49] WARNING: [pool www] child 17918, script '/home/site/public_html/index.php' (request: "GET /index.php") executing too slow (11.367854 sec), logging
[04-Jun-2013 21:24:49] NOTICE: child 17918 stopped for tracing
[04-Jun-2013 21:24:49] NOTICE: about to trace 17918
[04-Jun-2013 21:24:49] NOTICE: finished trace of 17918
[04-Jun-2013 21:24:53] WARNING: [pool www] child 18226, script '/home/site/public_html/index.php' (request: "GET /index.php") executing too slow (10.763667 sec), logging
[04-Jun-2013 21:24:53] WARNING: [pool www] child 18206, script '/home/site/public_html/index.php' (request: "GET /index.php") executing too slow (12.060464 sec), logging
[04-Jun-2013 21:24:53] WARNING: [pool www] child 18073, script '/home/site/public_html/index.php' (request: "GET /index.php") executing too slow (11.846097 sec), logging
[04-Jun-2013 21:24:53] NOTICE: child 18073 stopped for tracing
[04-Jun-2013 21:24:53] NOTICE: about to trace 18073
[04-Jun-2013 21:24:53] NOTICE: finished trace of 18073
[04-Jun-2013 21:24:53] NOTICE: child 18206 stopped for tracing
[04-Jun-2013 21:24:53] NOTICE: about to trace 18206
[04-Jun-2013 21:24:53] NOTICE: finished trace of 18206
[04-Jun-2013 21:24:53] NOTICE: child 18226 stopped for tracing
[04-Jun-2013 21:24:53] NOTICE: about to trace 18226
[04-Jun-2013 21:24:53] NOTICE: finished trace of 18226
[04-Jun-2013 21:24:56] WARNING: [pool www] child 5137, script '/home/site/public_html/index.php' (request: "GET /index.php") executing too slow (12.055624 sec), logging
[04-Jun-2013 21:24:56] WARNING: [pool www] child 18206, script '/home/site/public_html/index.php' (request: "GET /index.php") execution timed out (15.395149 sec), terminating
[04-Jun-2013 21:24:56] WARNING: [pool www] child 17996, script '/home/site/public_html/index.php' (request: "GET /index.php") executing too slow (12.145728 sec), logging
[04-Jun-2013 21:24:56] WARNING: [pool www] child 17918, script '/home/site/public_html/index.php' (request: "GET /index.php") execution timed out (18.036700 sec), terminating
[04-Jun-2013 21:24:56] NOTICE: child 17996 stopped for tracing
[04-Jun-2013 21:24:56] NOTICE: about to trace 17996
[04-Jun-2013 21:24:56] NOTICE: finished trace of 17996
[04-Jun-2013 21:24:56] NOTICE: child 5137 stopped for tracing
[04-Jun-2013 21:24:56] NOTICE: about to trace 5137
[04-Jun-2013 21:24:56] NOTICE: finished trace of 5137
[04-Jun-2013 21:24:56] WARNING: [pool www] child 17918 exited on signal 15 (SIGTERM) after 4424.343036 seconds from start
[04-Jun-2013 21:24:56] NOTICE: [pool www] child 6706 started
[04-Jun-2013 21:24:56] WARNING: [pool www] child 18206 exited on signal 15 (SIGTERM) after 4424.264130 seconds from start
[04-Jun-2013 21:24:56] NOTICE: [pool www] child 6707 started
[04-Jun-2013 21:24:59] WARNING: [pool www] child 17996, script '/home/site/public_html/index.php' (request: "GET /index.php") execution timed out (15.479201 sec), terminating
[04-Jun-2013 21:24:59] WARNING: [pool www] child 17996 exited on signal 15 (SIGTERM) after 4427.655572 seconds from start
[04-Jun-2013 21:24:59] NOTICE: [pool www] child 6708 started

这是我的nginx配置

user www-data;
worker_processes 4;
pid /var/run/nginx.pid;
worker_rlimit_nofile 20000;

events {
    #worker_connections 768;
    #worker_connections 19000;

    #multi_accept on;
    use epoll;
    #worker_connections 10240;  
    worker_connections 4096;
}


http {

    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    #keepalive_timeout 65;
    #keepalive_timeout 5;

#added
    client_body_timeout   15;
    client_header_timeout 15;
    keepalive_timeout     15;
    send_timeout          15;

site.conf

proxy_buffer_size   128k;
    proxy_buffers   4 256k;
    proxy_busy_buffers_size   256k;

    fastcgi_connect_timeout 60;
    fastcgi_send_timeout 180;
    fastcgi_read_timeout 180;
    fastcgi_buffer_size 128k;
    fastcgi_buffers 4 256k;
    #fastcgi_buffers 256 16k; #4096k total
    fastcgi_busy_buffers_size 256k;
    fastcgi_temp_file_write_size 256k;
    fastcgi_intercept_errors on;

php5-fpm通过TCP端口连接。

谢谢

Answer 1

我认为您可能运行了太多并发的php进程，但是如果没有更多关于资源瓶颈所在的信息，就很难知道。我设想您可能受到磁盘IO和/或CPU的限制，并且您的所有并行PHP进程都在竞争这些进程，并使彼此慢下来。在某种程度上，进程切换的开销成为一个重要的因素，通过运行大量的进程可以获得更少的吞吐量而不是更多的吞吐量。你也可能陷入或冒险，你耗尽了内存，开始交换，这是非常糟糕的。信任nginx能够对请求进行排队，并保持较高的处理速度，同时减少同时执行的请求。

我通常会选择从5到50个PHP进程，这个范围的两端都有一些特殊之处。更多的是10-15。对于非常高性能的磁盘系统，以及比通常的16个左右的核心，可能有更多的进程可能是有意义的，但与拥有更多更便宜的服务器相比，这通常是一种虚假的经济。根据我的经验，除非您的代码编写得很糟糕，否则在一台服务器上并行处理超过15个php进程通常没有什么好处，如果这样做的好处可能是稳定性，而不是吞吐量，面对长期运行的请求堆积如山，没有多余的进程可用。

如果您有多个具有独立流程池的代码库，那么您可能需要大量的进程，但是您可能不希望每个池的进程超过3到5个。

您确实需要许多处理静态文件的nginx工作人员连接。4096以后不太可能有任何改善，只有在不寻常的情况下，你才会看到1000和4000之间的差别。(除非您主要是为静态文件服务-这是一个完全不同的场景，但是由于您在这个框上讨论的是php进程，我不认为这里是这样的)。

我怀疑你的暂停时间太长了。如果没有任何事情发生，放下连接，进入下一个连接。

Answer 2

1)内存--我首先要看的是，如果脚本所做的只是一个简单的搜索，那么为什么它们需要50‘m的内存?我假设如果您每秒为数百个请求提供服务，那么每个用户不会实际返回多兆字节的数据。

有一个MySQL连接器库中的错误使PHP为任何文本或BLOB分配最大的大小，而不仅仅是实际需要的内存量。这可以通过移动到MySQLND库来解决，而不需要进行代码更改。

2)设置pm.max_requests = 10000可能不是一个很好的选择。如果每个请求花费2秒，那么您将告诉流程管理器在20,000秒或几乎6小时后重新启动每个进程。这似乎是一段很长的时间，而且对于任何内存泄漏都是足够的时间来破坏进程。将其恢复到500仍然只是每15分钟重新启动一次，这将不会影响性能，但可能会更稳定。

3)正如Michael所说，即使您能够允许尽可能多的进程与用户连接，您仍然需要弄清楚瓶颈到底在哪里。即使一次有多个PHP进程，如果它们都在等待SQL服务器可用，那么它们总是排队等待等待，并最终开始超时。

除非您能够消除瓶颈，否则您将需要实现一个速率限制机制，以便只允许服务器设置能够处理的多个查询，或者是拒绝服务器当前无法处理的请求的优雅降级。

Answer 3

如果其他一切都失败了..。我想也许你可以用密码来处理这件事。您可以创建一个“票务系统”，允许同时进行一定数量的搜索，并给用户一个大致的等待时间。比如“你的搜索将在N秒内开始”。