CeleryD似乎忽略了并发参数。

Question

最近，我将Django项目升级为芹菜4.4.6，而且进展不顺利。我目前的首要问题是任务的并发性。由于任务锁定数据库表，而且有些任务占用大量内存，因此不可能同时运行8个任务。我也只有一台2处理器的机器可用.然而，这正是芹菜所要做的。

以前，我只能同时运行两个任务。

工作人员被守护，并且只有一个工作人员处于活动状态(一个节点)。我把并发设置为2。这是我的/etc/default/celeryd：

#   most people will only start one node:
CELERYD_NODES="worker1"
#   but you can also start multiple and configure settings
#   for each in CELERYD_OPTS
#CELERYD_NODES="worker1 worker2 worker3"
#   alternatively, you can specify the number of nodes to start:
#CELERYD_NODES=3

# Absolute or relative path to the 'celery' command:
CELERY_BIN="/home/ubuntu/dev/bin/python -m celery"
#CELERY_BIN="/virtualenvs/def/bin/celery"

# App instance to use
# comment out this line if you don't use an app
CELERY_APP="match2"
# or fully qualified:
#CELERY_APP="proj.tasks:app"

# Where to chdir at start.
export DJANGO_SETTINGS_MODULE="match2.settings"
CELERYD_CHDIR="/home/ubuntu/dev/match2/match2"

# Extra command-line arguments to the worker
CELERYD_OPTS="--concurrency=2"
# Configure node-specific settings by appending node name to arguments:
#CELERYD_OPTS="--time-limit=300 -c 8 -c:worker2 4 -c:worker3 2 -Ofair:worker1"

# Set logging level to DEBUG
CELERYD_LOG_LEVEL="INFO"

# %n will be replaced with the first part of the nodename.
CELERYD_LOG_FILE="/var/log/celery/%n%I.log"
CELERYD_PID_FILE="/var/run/celery/%n.pid"

# Workers should run as an unprivileged user.
#   You need to create this user manually (or you can choose
#   a user/group combination that already exists (e.g., nobody).
CELERYD_USER="ubuntu"
CELERYD_GROUP="users"

# If enabled pid and log directories will be created if missing,
# and owned by the userid/group configured.
CELERY_CREATE_DIRS=1

我在很大程度上假设这一行可以同时执行多少任务：CELERYD_OPTS="--concurrency=2"，但它似乎仍然从RabbitMQ消息队列中提取最多8个项。

任何帮助都很感激。

Answer 1

因此，在反复加入谷歌集团之后，我终于得到了一个答案：

如果你想要芹菜表现得像一个好的小工人，在完成旧的任务之前不要承担另一项任务，你需要在你的设置文件中同时包含这两项任务：

task_acks_late = True
worker_prefetch_multiplier = 1

如果然后在Django项目中使用它，并使用旧样式的超感知设置(请参阅：https://docs.celeryproject.org/en/stable/userguide/configuration.html#new-lowercase-settings)，则可以将其转换为：

CELERY_WORKER_PREFETCH_MULTIPLIER = 1
CELERY_TASK_ACKS_LATE = True