NodeJS和Redis集群-错误:连接EMFILE

Question

我把头撞在墙上三天后就开始数了。我希望你能告诉我我做错了什么。

我有一个由三个主人和三个奴隶组成的红宝石群。NodeJS应用程序希望使用redis快速存储和服务数据。一切似乎都很好，然而，我们偶尔遇到申请下降。结果，它经常重新启动自己。重新启动时，应用程序(而不是redis)记录下列消息

2020-08-05 08:48 +02:00: events.js:288
2020-08-05 08:48 +02:00:       throw er; // Unhandled 'error' event
2020-08-05 08:48 +02:00:       ^
2020-08-05 08:48 +02:00: 
2020-08-05 08:48 +02:00: Error: connect EMFILE 10.0.0.201:7101 - Local (undefined:undefined)
2020-08-05 08:48 +02:00:     at internalConnect (net.js:917:16)
2020-08-05 08:48 +02:00:     at defaultTriggerAsyncIdScope (internal/async_hooks.js:311:12)
2020-08-05 08:48 +02:00:     at net.js:1007:9
2020-08-05 08:48 +02:00:     at processTicksAndRejections (internal/process/task_queues.js:79:11)
2020-08-05 08:48 +02:00:     at runNextTicks (internal/process/task_queues.js:66:3)
2020-08-05 08:48 +02:00:     at listOnTimeout (internal/timers.js:518:9)
2020-08-05 08:48 +02:00:     at processTimers (internal/timers.js:492:7)
2020-08-05 08:48 +02:00: Emitted 'error' event on Socket instance at:
2020-08-05 08:48 +02:00:     at emitErrorNT (internal/streams/destroy.js:92:8)
2020-08-05 08:48 +02:00:     at emitErrorAndCloseNT (internal/streams/destroy.js:60:3)
2020-08-05 08:48 +02:00:     at processTicksAndRejections (internal/process/task_queues.js:84:21)
2020-08-05 08:48 +02:00:     at runNextTicks (internal/process/task_queues.js:66:3)
2020-08-05 08:48 +02:00:     at listOnTimeout (internal/timers.js:518:9)
2020-08-05 08:48 +02:00:     at processTimers (internal/timers.js:492:7) {
2020-08-05 08:48 +02:00:   errno: 'EMFILE',
2020-08-05 08:48 +02:00:   code: 'EMFILE',
2020-08-05 08:48 +02:00:   syscall: 'connect',
2020-08-05 08:48 +02:00:   address: '10.0.0.201',
2020-08-05 08:48 +02:00:   port: 7101
2020-08-05 08:48 +02:00: }
2020-08-05 08:48 +02:00: events.js:288
2020-08-05 08:48 +02:00:       throw er; // Unhandled 'error' event
2020-08-05 08:48 +02:00:       ^
2020-08-05 08:48 +02:00: 
2020-08-05 08:48 +02:00: Error [ERR_IPC_CHANNEL_CLOSED]: Channel closed
2020-08-05 08:48 +02:00:     at ChildProcess.target.send (internal/child_process.js:679:16)
2020-08-05 08:48 +02:00:     at Worker.send (internal/cluster/worker.js:45:28)
2020-08-05 08:48 +02:00:     at Server.<anonymous> (/totally/secret/folder/structure/app.js:136:16)
2020-08-05 08:48 +02:00:     at Server.emit (events.js:311:20)
2020-08-05 08:48 +02:00:     at TCP.onconnection (net.js:1554:8)
2020-08-05 08:48 +02:00: Emitted 'error' event on Worker instance at:
2020-08-05 08:48 +02:00:     at ChildProcess.<anonymous> (internal/cluster/worker.js:29:12)
2020-08-05 08:48 +02:00:     at ChildProcess.emit (events.js:311:20)
2020-08-05 08:48 +02:00:     at internal/child_process.js:683:35
2020-08-05 08:48 +02:00:     at processTicksAndRejections (internal/process/task_queues.js:79:11) {
2020-08-05 08:48 +02:00:   code: 'ERR_IPC_CHANNEL_CLOSED'
2020-08-05 08:48 +02:00: }
2020-08-05 08:48 +02:00: WARNING: NODE_APP_INSTANCE value of '0' did not match any instance config file names.
2020-08-05 08:48 +02:00: WARNING: See https://github.com/lorenwest/node-config/wiki/Strict-Mode
2020-08-05 08:48 +02:00: WARNING: NODE_APP_INSTANCE value of '0' did not match any instance config file names.
2020-08-05 08:48 +02:00: WARNING: See https://github.com/lorenwest/node-config/wiki/Strict-Mode
2020-08-05 08:48 +02:00: WARNING: NODE_APP_INSTANCE value of '0' did not match any instance config file names.
2020-08-05 08:48 +02:00: WARNING: See https://github.com/lorenwest/node-config/wiki/Strict-Mode
2020-08-05 08:48 +02:00: WARNING: NODE_APP_INSTANCE value of '0' did not match any instance config file names.
2020-08-05 08:48 +02:00: WARNING: See https://github.com/lorenwest/node-config/wiki/Strict-Mode
2020-08-05 08:48 +02:00: WARNING: NODE_APP_INSTANCE value of '0' did not match any instance config file names.
2020-08-05 08:48 +02:00: WARNING: See https://github.com/lorenwest/node-config/wiki/Strict-Mode
2020-08-05 08:48 +02:00: (node:28699) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'emit' of undefined
2020-08-05 08:48 +02:00:     at /totally/secret/folder/structure/app.js:323:34
2020-08-05 08:48 +02:00:     at runMicrotasks (<anonymous>)
2020-08-05 08:48 +02:00:     at processTicksAndRejections (internal/process/task_queues.js:97:5)
2020-08-05 08:48 +02:00: (node:28699) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 530)
2020-08-05 08:48 +02:00: (node:28699) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
2020-08-05 08:48 +02:00: (node:28699) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 201 error listeners added to [Cluster]. Use emitter.setMaxListeners() to increase limit

到目前为止，我尝试的是禁用AOF或RDB，但是没有成功，偶尔还是会重新启动。这个问题的起因还会是什么呢？开发人员说，应用程序根本无法连接到Redis，因此失败了。但这不可能。点击服务器并查看服务的正常运行时间确认了100%，即服务器本身始终是可访问的，并可供应用程序使用。所有六个人。

Answer 1

EMFILE的错误是“打开的文件太多”。在打开网络连接的过程中，需要一个文件描述符，因此由于您同时打开了数千个连接，所以您正在运行打开的文件ulimit，默认为1024。

您可以通过在MAX_OPEN_FILES=中设置/etc/default/pm2并重新启动pm2和应用程序来解决这个问题。

MAX_OPEN_FILES=8192