当尝试将雪花与火花放电连接时，获得未解决的依赖关系错误

Question

试图连接雪花数据库使用Pysaprk。下面是我的密码。

from pyspark.sql import SparkSession
import os

os.environ['PYSPARK_SUBMIT_ARGS'] = '--packages net.snowflake:snowflake-jdbc:3.13.14,net.snowflake:spark-snowflake_2.12:2.10.0-spark_3.2 pyspark-shell'

sfoptions = {
"sfUrl" : URL,
"sfUser" : usr,
"sfPassword" : paswd,
"sfAccount" : account,
"sfDatabase" : database,
"sfSchema" : schema,
"sfWarehouse" : warehouse,
"sfRole" : role,
}
#
query="select count(*) from table"

spark = SparkSession.builder.appName("demo").master("local").getOrCreate()

SNOWFLAKE_SOURCE_NAME="net.snowflake.spark.snowflake"    
df=spark.read.format(SNOWFLAKE_SOURCE_NAME).options(**sfoptions).option("query",query).load()
df.show()

但如果你犯了错误，

net.snowflake#snowflake-jdbc added as a dependency
net.snowflake#spark-snowflake_2.12 added as a dependency
:: resolving dependencies :: org.apache.spark#spark-submit-parent-2cb3619a-01c7-4bb3-b74e-ec747c450381;1.0
    confs: [default]
You probably access the destination server through a proxy server that is not well configured.
You probably access the destination server through a proxy server that is not well configured.
You probably access the destination server through a proxy server that is not well configured.
You probably access the destination server through a proxy server that is not well configured.
You probably access the destination server through a proxy server that is not well configured.
You probably access the destination server through a proxy server that is not well configured.
You probably access the destination server through a proxy server that is not well configured.
You probably access the destination server through a proxy server that is not well configured.
:: resolution report :: resolve 543ms :: artifacts dl 1ms
    :: modules in use:
    ---------------------------------------------------------------------
    |                  |            modules            ||   artifacts   |
    |       conf       | number| search|dwnlded|evicted|| number|dwnlded|
    ---------------------------------------------------------------------
    |      default     |   2   |   0   |   0   |   0   ||   0   |   0   |
    ---------------------------------------------------------------------

:: problems summary ::
:::: WARNINGS
    Host repo1.maven.org not found. url=https://repo1.maven.org/maven2/net/snowflake/snowflake-jdbc/3.13.14/snowflake-jdbc-3.13.14.pom

    Host repo1.maven.org not found. url=https://repo1.maven.org/maven2/net/snowflake/snowflake-jdbc/3.13.14/snowflake-jdbc-3.13.14.jar

    Host repos.spark-packages.org not found. url=https://repos.spark-packages.org/net/snowflake/snowflake-jdbc/3.13.14/snowflake-jdbc-3.13.14.pom

    Host repos.spark-packages.org not found. url=https://repos.spark-packages.org/net/snowflake/snowflake-jdbc/3.13.14/snowflake-jdbc-3.13.14.jar

        module not found: net.snowflake#snowflake-jdbc;3.13.14

它说的是“未解决的依赖”。它试图访问不可用的存储库-火花-Packages.org。请帮我解决这个问题。如何从python代码中添加/提到雪花和火花连接器？

::::::::::::::::::::::::::::::::::::::::::::::

        ::          UNRESOLVED DEPENDENCIES         ::

        ::::::::::::::::::::::::::::::::::::::::::::::

        :: net.snowflake#snowflake-jdbc;3.13.14: not found

        :: net.snowflake#spark-snowflake_2.12;2.10.0-spark_3.2: not found

        ::::::::::::::::::::::::::::::::::::::::::::::

Answer 1

这似乎是一个网络问题，您目前的主机没有互联网访问，也许您应该检查您的系统管理员，以打开互联网连接。

您可以使用以下命令进行检查：

ping repo1.maven.org