Databricks连接:无法从IntelliJ运行databricks集群中的scala程序

Question

遵循这个医生中提到的步骤。databricks-connect test命令工作正常。但是，当我从Intellij启动测试scala程序时，我看到以下错误：

Exception in thread "main" java.lang.NoSuchMethodError: com.databricks.spark.util.MetricDefinitions$.EVENT_INITIAL_CONFIG_LOG()Lcom/databricks/spark/util/MetricDefinition;
    at com.databricks.spark.util.InitialConfigLogging$.recordInitialConfigs(InitialConfigLogging.scala:47)
    at org.apache.spark.sql.internal.SQLConf.recordSessionInitialConfs(SQLConf.scala:4166)
    at org.apache.spark.sql.SparkSession.<init>(SparkSession.scala:150)
    at org.apache.spark.sql.SparkSession$Builder.getOrCreate(SparkSession.scala:1033)
    at com.srikanth.Demo$.main(Demo.scala:13)
    at com.srikanth.Demo.main(Demo.scala)

环境详情：

python - 3.8
java - 1.8
databricks-connect - 9.1

Answer 1

我有同样的错误信息，结果我没有正确地添加databricks-connect库。您应该添加如下所述的库：https://docs.databricks.com/dev-tools/databricks-connect.html#intellij-scala-or-java