使用apache-气流提供商-气流上的雪花(没有模块名为雪花)

Question

我已经在码头上的气流上安装了apache-airflow-providers-snowflake软件包，而且我收到了错误。

没有名为雪花的模块--请参考附件(请检查提到的雪花钩子文件的错误)

经过分析，我发现雪花钩子操作符源代码是用文件名snowflake.py保存的，并在源代码中尝试导入雪花模块。因此，基本上，它是试图导入自己。

import logging

import airflow
import datetime
from airflow import DAG
#from snowflake.connector import *
#import snowflake.connector
from airflow.operators.python import PythonOperator
from airflow.providers.snowflake.hooks.snowflake import SnowflakeHook


logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)

args = {"owner": "Airflow", "start_date": datetime.datetime.now()}

dag = DAG(
    "snowflake_poc", default_args=args, schedule_interval=None
)

#create_insert_query = [
#    """INSERT INTO STRUCTURED.error_details_log (SELECT PARSE_JSON('{"row_num": "13456"}'),current_date());""",
#]


def row_count(**context):
    dwh_hook = SnowflakeHook(snowflake_conn_id="snowflake_conn")
    result = dwh_hook.get_first("select count(*) from STRUCTURED.LKP_GENDER")
    print(result)
    logging.info("Number of rows in `STRUCTURED.LKP_GENDER`  - %s", result[0])


with dag:
    get_count = PythonOperator(task_id="get_count", python_callable=row_count)
get_count

$ pip freeze
adal==1.2.6
alembic==1.5.5
amqp==2.6.1
apache-airflow @ file:///opt/airflow
apache-airflow-providers-snowflake==1.1.0
apispec==3.3.2
appdirs==1.4.4
argcomplete==1.12.2
asn1crypto==1.4.0
attrs==20.3.0
azure-batch==10.0.0
azure-common==1.1.26
azure-core==1.11.0
azure-cosmos==3.2.0
azure-datalake-store==0.0.51
azure-identity==1.5.0
azure-keyvault==4.1.0
azure-keyvault-certificates==4.2.1
azure-keyvault-keys==4.3.1
azure-keyvault-secrets==4.2.0
azure-kusto-data==0.0.45
azure-mgmt-containerinstance==1.5.0
azure-mgmt-core==1.2.2
azure-mgmt-datalake-nspkg==3.0.1
azure-mgmt-datalake-store==0.5.0
azure-mgmt-nspkg==3.0.2
azure-mgmt-resource==15.0.0
azure-nspkg==3.0.2
azure-storage-blob==12.7.1
azure-storage-common==2.1.0
azure-storage-file==2.1.0
Babel==2.9.0
bcrypt==3.2.0
billiard==3.6.3.0
boto3==1.15.18
botocore==1.18.18
cached-property==1.5.2
cachetools==4.2.1
cattrs==1.2.0
celery==4.4.7
certifi==2020.12.5
cffi==1.14.5
chardet==3.0.4
click==7.1.2
clickclick==20.10.2
cloudpickle==1.4.1
colorama==0.4.4
colorlog==4.7.2
commonmark==0.9.1
connexion==2.7.0
croniter==0.3.37
cryptography==3.4.6
dask==2021.2.0
defusedxml==0.6.0
dill==0.3.3
distlib==0.3.1
distributed==2.19.0
dnspython==1.16.0
docker==3.7.3
docker-pycreds==0.4.0
docutils==0.16
elasticsearch==7.5.1
elasticsearch-dbapi==0.1.0
elasticsearch-dsl==7.3.0
email-validator==1.1.2
eventlet==0.30.1
filelock==3.0.12
Flask==1.1.2
Flask-AppBuilder==3.1.1
Flask-Babel==1.0.0
Flask-Caching==1.9.0
Flask-JWT-Extended==3.25.1
Flask-Login==0.4.1
Flask-OpenID==1.2.5
Flask-SQLAlchemy==2.4.4
Flask-WTF==0.14.3
flower==0.9.7
gevent==21.1.2
google-ads==7.0.0
google-api-core==1.26.0
google-api-python-client==1.12.8
google-auth==1.27.0
google-auth-httplib2==0.0.4
google-auth-oauthlib==0.4.2
google-cloud-automl==2.1.0
google-cloud-bigquery==2.9.0
google-cloud-bigquery-datatransfer==3.0.0
google-cloud-bigquery-storage==2.3.0
google-cloud-bigtable==1.7.0
google-cloud-container==1.0.1
google-cloud-core==1.6.0
google-cloud-datacatalog==3.0.0
google-cloud-dataproc==2.2.0
google-cloud-dlp==1.0.0
google-cloud-kms==2.2.0
google-cloud-language==1.3.0
google-cloud-logging==2.2.0
google-cloud-memcache==0.3.0
google-cloud-monitoring==2.0.1
google-cloud-os-login==2.1.0
google-cloud-pubsub==2.3.0
google-cloud-redis==2.0.0
google-cloud-secret-manager==1.0.0
google-cloud-spanner==1.19.1
google-cloud-speech==1.3.2
google-cloud-storage==1.36.0
google-cloud-tasks==2.1.0
google-cloud-texttospeech==1.0.1
google-cloud-translate==1.7.0
google-cloud-videointelligence==1.16.1
google-cloud-vision==1.0.0
google-cloud-workflows==0.2.0
google-crc32c==1.1.2
google-resumable-media==1.2.0
googleapis-common-protos==1.52.0
graphviz==0.16
greenlet==1.0.0
grpc-google-iam-v1==0.12.3
grpcio==1.35.0
grpcio-gcp==0.2.2
gunicorn==19.10.0
HeapDict==1.0.1
httplib2==0.19.0
humanize==3.2.0
hvac==0.10.8
idna==2.10
importlib-metadata==1.7.0
importlib-resources==1.5.0
inflection==0.5.1
iso8601==0.1.14
isodate==0.6.0
itsdangerous==1.1.0
Jinja2==2.11.3
jmespath==0.10.0
json-merge-patch==0.2
jsonschema==3.2.0
kombu==4.6.11
kubernetes==11.0.0
lazy-object-proxy==1.4.3
ldap3==2.9
libcst==0.3.17
lockfile==0.12.2
Mako==1.1.4
Markdown==3.3.3
MarkupSafe==1.1.1
marshmallow==3.10.0
marshmallow-enum==1.5.1
marshmallow-oneofschema==2.1.0
marshmallow-sqlalchemy==0.23.1
msal==1.9.0
msal-extensions==0.3.0
msgpack==1.0.2
msrest==0.6.21
msrestazure==0.6.4
mypy-extensions==0.4.3
mysql-connector-python==8.0.22
mysqlclient==1.3.14
natsort==7.1.1
numpy==1.20.1
oauthlib==2.1.0
openapi-spec-validator==0.2.9
oscrypto==1.2.1
packaging==20.9
pandas==1.2.2
pandas-gbq==0.14.1
paramiko==2.7.2
pendulum==2.1.2
portalocker==1.7.1
prison==0.1.3
prometheus-client==0.8.0
proto-plus==1.13.0
protobuf==3.15.1
psutil==5.8.0
psycopg2-binary==2.8.6
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
pycryptodomex==3.10.1
pydata-google-auth==1.1.0
Pygments==2.8.0
PyJWT==1.7.1
PyNaCl==1.4.0
pyOpenSSL==19.1.0
pyparsing==2.4.7
pyrsistent==0.17.3
pysftp==0.2.9
python-daemon==2.3.0
python-dateutil==2.8.1
python-editor==1.0.4
python-http-client==3.3.2
python-ldap==3.3.1
python-nvd3==0.15.0
python-slugify==4.0.1
python3-openid==3.2.0
pytz==2020.5
pytzdata==2020.1
PyYAML==5.4.1
redis==3.5.3
requests==2.25.1
requests-oauthlib==1.1.0
rich==9.2.0
rsa==4.7.1
s3transfer==0.3.4
sendgrid==6.6.0
setproctitle==1.2.2
six==1.15.0
slack-sdk==3.4.0
snowflake-connector-python==2.3.10
snowflake-sqlalchemy==1.2.4
sortedcontainers==2.3.0
SQLAlchemy==1.3.23
SQLAlchemy-JSONField==1.0.0
SQLAlchemy-Utils==0.36.8
sshtunnel==0.1.5
starkbank-ecdsa==1.1.0
statsd==3.3.0
swagger-ui-bundle==0.0.8
tabulate==0.8.8
tblib==1.7.0
tenacity==6.2.0
termcolor==1.1.0
text-unidecode==1.3
toolz==0.11.1
tornado==6.1
typing-extensions==3.7.4.3
typing-inspect==0.6.0
unicodecsv==0.14.1
uritemplate==3.0.1
urllib3==1.25.11
vine==1.3.0
virtualenv==20.4.2
watchtower==0.7.3
websocket-client==0.57.0
Werkzeug==1.0.1
WTForms==2.3.3
zict==2.0.0
zipp==3.4.0
zope.event==4.5.0
zope.interface==5.2.0

Answer 1

我正在导入下面的库及其与airflow 2.0.2和snowflake-connector-python==2.4.3的工作关系。

from airflow.providers.snowflake.operators.snowflake import SnowflakeOperator