Airflow x Bitbucket (使用Kubernetes Executor)

Question

我正在尝试使用Kubernetes Executor在单独的pod中处理我创建的dags！

我们有一个步骤是同步git来访问dags，首先我在airflow.cfg中放了以下参数，但在这些条件下我不能处理简单的python操作符(它不会创建pod)。有没有人能做到这一点？git sync的设置是否正确？

Airflow.cfg：

# For either git sync or volume mounted DAGs, the worker will look in this subpath for DAGs
dags_volume_subpath =

# For DAGs mounted via a volume claim (mutually exclusive with git-sync and host path)
dags_volume_claim =

# For volume mounted logs, the worker will look in this subpath for logs
logs_volume_subpath =

# A shared volume claim for the logs
logs_volume_claim =

# For DAGs mounted via a hostPath volume (mutually exclusive with volume claim and git-sync)
# Useful in local environment, discouraged in production
dags_volume_host =

# A hostPath volume for the logs
# Useful in local environment, discouraged in production
logs_volume_host =

# A list of configMapsRefs to envFrom. If more than one configMap is
# specified, provide a comma separated list: configmap_a,configmap_b
env_from_configmap_ref =

# A list of secretRefs to envFrom. If more than one secret is
# specified, provide a comma separated list: secret_a,secret_b
env_from_secret_ref =

# Git credentials and repository for DAGs mounted via Git (mutually exclusive with volume claim)
git_repo = git@bitbucket.org:myuser/airflow-kubernetes.git
git_branch = master
git_subpath = 

# Use git_user and git_password for user authentication or git_ssh_key_secret_name and git_ssh_key_secret_key
# for SSH authentication
git_user =
git_password =
git_sync_root = git@bitbucket.org:myuser/airflow-kubernetes.git
git_sync_dest = /airflow/dags

# Mount point of the volume if git-sync is being used.
# i.e. /airflow/dags
git_dags_folder_mount_point = /airflow/dags

​

​

​

​

Answer 1

我可以通过配置以下属性来实现这一点：

dags_in_image = False
dags_volume_subpath = dags
git_repo = https://github.com/<user-name>/<repo-name>.git
git_branch = master
git_subpath = dags
git_user = <git-username>
git_password = <git-password>
git_sync_root = /git
git_sync_dest = repo
git_dags_folder_mount_point = <path-to-dag-directory-locally>

我的存储库中有一个文件夹( dags )，dags存储在其中。