SingleStore火花连接器:读/写操作时NULLPointer异常

Question

读/写操作是从火花外壳工作的。但是在本地从Development执行时抛出NULLPointer异常。

   val df = spark.read
  .format("singlestore")
  .option("ddlEndpoint", "host:port")
  .option("user", "xxxxx")
  .option("password","xxxxx")
  .option("database","xxxxx")
  .load("schema.table_name")

我得到了以下错误：

Exception in thread "main" java.lang.NullPointerException
at com.singlestore.spark.JdbcHelpers$ConnectionHelpers.withStatement(JdbcHelpers.scala:26)
at com.singlestore.spark.JdbcHelpers$.getSinglestoreVersion(JdbcHelpers.scala:322)
at com.singlestore.spark.SQLGen$SQLGenContext$.getSinglestoreVersion(SQLGen.scala:532)
at com.singlestore.spark.SQLGen$SQLGenContext$.apply(SQLGen.scala:550)
at com.singlestore.spark.DefaultSource.createRelation(DefaultSource.scala:57)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:332)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:242)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:230)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:197)

下面的依赖项在sbt.build文件中设置。

libraryDependencies += "org.apache.spark" %% "spark-core" % "2.4.7"
libraryDependencies += "org.apache.spark" %% "spark-sql" % "2.4.7"
libraryDependencies += "com.singlestore" % "singlestore-spark-connector_2.11" % "3.1.2-spark-2.4.7"
libraryDependencies += "org.mariadb.jdbc" % "mariadb-java-client"  % "3.0.4"

有人能帮我解决这个问题吗。谢谢!

Answer 1

看起来，您需要更新sbt.build文件中的一个依赖项。

试着替换：

libraryDependencies += "org.mariadb.jdbc" % "mariadb-java-client"  % "3.0.4"

使用

libraryDependencies += "org.mariadb.jdbc" % "mariadb-java-client"  % "2.+"