在conda环境中运行气流并使用systemd无法启动

Question

我想通过conda安装气流，并使用systemd控制Ubuntu上的气流。我已经通过以下步骤成功地将气流安装到conda环境中，但是我还不能正确地配置systemd来处理气流。

详细信息：

• Ubuntu 18.04
• Python 3.7.4
• 气流1.10.4

我见过SO #52310217，但问题仍未解决。

下面是从命令行安装和运行气流的步骤：

$ conda create --name airflow -c conda-forge airflow
$ conda activate airflow
$ export AIRFLOW_HOME=~/airflow
$ airflow initdb

$ airflow scheduler
$ airflow webserver

在这里，我在/etc/systemd/system/中创建了以下文件

# /etc/systemd/system/airflow-scheduler.service
[Unit]
Description=Airflow scheduler daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
Environment="PATH=/home/ubuntu/python/envs/airflow/bin"
User=airflow
Group=airflow
Type=simple
ExecStart=/home/curtis/miniconda3/envs/airflow/bin/airflow scheduler
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target

# /etc/systemd/system/airflow-webserver.service
[Unit]
Description=Airflow webserver daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
Environment="PATH=/home/ubuntu/python/envs/airflow/bin"
User=airflow
Group=airflow
Type=simple
ExecStart=/home/curtis/miniconda3/envs/airflow/bin/airflow webserver -p 8085 --pid /home/curtis/airflow/airflow-webserver.pid
Restart=on-failure
RestartSec=5s
PrivateTmp=true

[Install]
WantedBy=multi-user.target

在创建上述文件之后，我发出了以下命令，但没有发生任何事情。

sudo systemctl daemon-reload
sudo systemctl enable airflow-scheduler
sudo systemctl start airflow-scheduler
sudo systemctl enable airflow-webserver
sudo systemctl start airflow-webserver

如果我在终端窗口中直接输入ExecStart命令，气流调度程序和气流and服务器将按预期的方式启动。因此，我认为这不是气流的问题，而是我在systemd配置中缺少的东西。

我不知道该从这里拿走什么，但这里是气流调度器和气流网络服务器的状态。

[curtis:~/airflow/logs/scheduler] [airflow] 4 $ sudo systemctl status airflow-scheduler
● airflow-scheduler.service - Airflow scheduler daemon
   Loaded: loaded (/etc/systemd/system/airflow-scheduler.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Mon 2019-08-19 22:36:35 PDT; 3s ago
  Process: 5623 ExecStart=/home/curtis/miniconda3/envs/airflow/bin/airflow scheduler (code=exited, status=217/USER)
 Main PID: 5623 (code=exited, status=217/USER)

[curtis:~/airflow/logs/scheduler] [airflow] 3 $ sudo systemctl status airflow-webserver
● airflow-webserver.service - Airflow webserver daemon
   Loaded: loaded (/etc/systemd/system/airflow-webserver.service; enabled; vendor preset: enabled)
   Active: activating (auto-restart) (Result: exit-code) since Mon 2019-08-19 22:36:46 PDT; 895ms ago
  Process: 5805 ExecStart=/home/curtis/miniconda3/envs/airflow/bin/airflow webserver -p 8085 --pid /home/curtis/airflow/airflow-webserver
 Main PID: 5805 (code=exited, status=217/USER)

Answer 1

我发现了这个问题，并想在这里为其他遇到同样问题的人发这篇文章。

• User应该是您当前的用户名，或者是安装气流的用户名。就我而言，那是“柯蒂斯”
• Group应该是您当前的组，或者是安装了气流的用户组。就我而言，那是“柯蒂斯”
• Environment是您的conda环境的路径。在我的例子中，那是PATH=/home/curtis/miniconda3/envs/airflow/bin

我已经包括了我的修正后的systemd文件供您参考。

# /etc/systemd/system/airflow-scheduler.service
[Unit]
Description=Airflow scheduler daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
Environment="PATH=/home/curtis/miniconda3/envs/airflow/bin"
User=curtis
Group=curtis
Type=simple
ExecStart=/home/curtis/miniconda3/envs/airflow/bin/airflow scheduler
Restart=always
RestartSec=5s

[Install]
WantedBy=multi-user.target

# /etc/systemd/system/airflow-webserver.service
[Unit]
Description=Airflow webserver daemon
After=network.target postgresql.service mysql.service redis.service rabbitmq-server.service
Wants=postgresql.service mysql.service redis.service rabbitmq-server.service

[Service]
Environment="PATH=/home/curtis/miniconda3/envs/airflow/bin"
User=curtis
Group=curtis
Type=simple
ExecStart=/home/curtis/miniconda3/envs/airflow/bin/airflow webserver -p 8085 --pid /home/curtis/airflow/airflow-webserver.pid
Restart=on-failure
RestartSec=5s
PrivateTmp=true

[Install]
WantedBy=multi-user.target