Scala模块2.8.11需要杰克逊数据库版本>= 2.8.0和< 2.9.0

Question

对于AWS胶水作业，我使用Scala2.11和Spark2.4.3。最近，我在我们的构建管道中得到了下面的错误消息。

原因: com.fasterxml.jackson.databind.JsonMappingException: Scala模块2.8.11需要版本>= 2.8.0和< 2.9.0

我试过-

1. 将jackson-模块-scala版本从2.8.11更改为2.12.0。这修复了构建管道，但我在胶水作业

中得到了另一条错误消息。

用户类中的异常: java.lang.ExceptionInInitializerError。

1. 将jackson模块-scala版本从2.8.11更改为2.13.1。我重构代码以获得单元测试，并修复构建管道，但是在胶水作业中，在

下面得到错误消息。

User类中的异常: java.lang.VerifyError :错误的返回类型

1. 我尝试过在jackson ScalaVersion2.12.0中添加dependencyOverrides。构建管道可以工作，但胶水作业将失败。

dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-core" % "2.8.7"
dependencyOverrides += "com.fasterxml.jackson.core" % "jackson-databind" % "2.8.7"
dependencyOverrides += "com.fasterxml.jackson.module" % "jackson-module-scala_2.11" % "2.8.7"

你知道我做错了什么或者怎么解决我的问题吗？

有关build.sbt文件，请参见下面的

name in ThisBuild := "etl"
organization in ThisBuild := "XXXXXXXXXX"
scalaVersion in ThisBuild := "2.11.12"
version in ThisBuild := "0.2"

addCommandAlias("sanity", ";clean ;compile ;test ;scalafmtAll ;scalastyle ;assembly")

lazy val framework = project.settings(settings, libraryDependencies ++= commonDependencies)

lazy val scripts = project
  .settings(settings, libraryDependencies ++= commonDependencies)
  .dependsOn(framework % "compile->compile;test->test")

lazy val settings = Seq(
  test in assembly := {},
  scalacOptions ++= Seq(),
  resolvers ++= Seq(
    Resolver.sonatypeRepo("releases"),
    "aws-glue-etl-artifacts" at "https://aws-glue-etl-artifacts.s3.amazonaws.com/release/"
  ),
  assemblyMergeStrategy in assembly := {
    case PathList("META-INF", "io.netty.versions.properties", xs @ _*) => MergeStrategy.singleOrError
    case "module-info.class" => MergeStrategy.discard
    case x: String if x.contains("UnusedStubClass") => MergeStrategy.first
    case y =>
      val oldStrategy = (assemblyMergeStrategy in assembly).value
      oldStrategy(y)
  }
)

lazy val commonDependencies = Seq(
  "com.amazonaws" % "AWSGlueETL" % "1.0.0" % Provided,
  "com.databricks" %% "spark-xml" % "0.8.0",
  "org.scalatest" %% "scalatest" % "3.1.1" % Test,
  "org.scalamock" %% "scalamock" % "4.4.0" % Test,
  "com.fasterxml.jackson.module" %% "jackson-module-scala" % "2.8.11",
  "io.netty" % "netty-all" % "4.1.17.Final" % Test, 
  "org.apache.spark" %% "spark-avro" % "2.4.3",
  "com.crealytics" %% "spark-excel" % "0.13.6"
)


fork in ThisBuild := true
parallelExecution in ThisBuild := true
testForkedParallel in ThisBuild := false
logBuffered in ThisBuild := false
testOptions in ThisBuild += Tests.Argument(TestFrameworks.ScalaTest, "-oDFG")

javaOptions ++= Seq(
  "-XX:+CMSClassUnloadingEnabled",
  "-XX:MaxMetaspaceSize=512M",
  "-XX:MetaspaceSize=256M",
  "-Xms512M",
  "-Xmx2G",
  "-XX:MaxPermSize=2048M"
)

下面的函数使用Jackson模块scala 2.8.11 (最多2.12.0)

import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper

object JsonUtils {
  private val mapper = new ObjectMapper() with ScalaObjectMapper
  mapper.registerModule(DefaultScalaModule)

  def fromJson[T](json: String)(implicit m: Manifest[T]): T = {
    mapper.readValue[T](json)
  }
}

Answer 1

你需要重写它，像这样：

dependencyOverrides ++= Seq(
"com.fasterxml.jackson.core" % "jackson-databind" % versions("jackson"),
"com.fasterxml.jackson.core" % "jackson-core" % versions("jackson"))

Answer 2

我已经解决了我的问题。

在函数中，我将库导入更改为

import com.fasterxml.jackson.module.scala.experimental.ScalaObjectMapper

至

import com.fasterxml.jackson.module.scala.ScalaObjectMapper

在build.sbt中，我添加了以下几行

libraryDependencies += "com.fasterxml.jackson.module" % "jackson-module-scala_2.11" % "2.12.0"

dependencyOverrides += "com.fasterxml.jackson.module" % "jackson-module-scala_2.11" % "2.6.7.1"