我一直在尝试将一个maven项目转换为一个级别构建。POM.xml中的依赖项之一是avro-tools,如下所示:
<dependency>
<groupId>org.apache.avro</groupId>
<artifactId>avro-tools</artifactId>
<version>1.10.2</version>
</dependency>在项目中运行gradle init时,gradle将从build.gradle文件中生成一个pom.xml文件,其依赖项如下:实现组:'org.apache.avro',名称:'avro-tools',版本:‘1.10.0’
但是,在运行构建时,我会得到以下错误:
Could not find avro-mapred-1.10.2-hadoop2.jar (org.apache.avro:avro-mapred:1.10.2).
Searched in the following locations:
https://repo.maven.apache.org/maven2/org/apache/avro/avro-mapred/1.10.2/avro-mapred-1.10.2-hadoop2.jar
Possible solution:
- Declare repository providing the artifact, see the documentation at https://docs.gradle.org/current/userguide/declaring_repositories.html如果在maven中运行相同的构建,则构建可以正常工作。
编辑:
完整的build.gradle文件:
plugins {
id 'java'
}
group 'org.example'
version '1.0-SNAPSHOT'
repositories {
mavenCentral()
}
dependencies {
implementation group: 'org.scala-lang', name: 'scala-library', version:'2.12.6'
implementation group: 'org.apache.spark', name: 'spark-core_2.12', version:'2.4.1'
implementation group: 'org.apache.spark', name: 'spark-yarn_2.12', version:'2.4.1'
implementation group: 'org.apache.spark', name: 'spark-sql_2.12', version:'2.4.1'
implementation group: 'org.apache.spark', name: 'spark-hive_2.12', version:'2.4.1'
implementation group: 'org.apache.spark', name: 'spark-avro_2.12', version:'2.4.1'
implementation group: 'com.typesafe', name: 'config', version:'1.4.1'
implementation group: 'com.sun.mail', name: 'javax.mail', version:'1.5.6'
implementation group: 'org.apache.avro', name: 'avro-tools', version:'1.10.2'
implementation group: 'com.typesafe.play', name: 'play-json_2.12', version:'2.9.2'
testImplementation group: 'junit', name: 'junit', version:'4.12'
testImplementation group: 'org.scalatest', name: 'scalatest_2.12', version:'3.0.5'
testImplementation group: 'org.specs2', name: 'specs2-core_2.12', version:'4.2.0'
testImplementation group: 'org.specs2', name: 'specs2-junit_2.12', version:'4.2.0'
testImplementation group: 'org.scalatestplus', name: 'mockito-3-4_2.12', version:'3.2.4.0'
testImplementation group: 'org.mockito', name: 'mockito-core', version:'3.4.2'
testImplementation group: 'org.mockito', name: 'mockito-inline', version:'4.0.0'
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.7.0'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.7.0'
}
test {
useJUnitPlatform()
}编辑:解决了问题,这不仅仅是avro工具,而是avro工具&导致依赖性冲突的火花核心2.12插件。avro-tools 10.2引用avro-mapred 1.10.2作为一个传递依赖项,而spark 2.12引用avro映射1.8.2作为依赖项。有人知道怎么解决这个问题吗?
发布于 2022-09-19 06:09:03
虽然我面对的问题与不同的图书馆,海事组织的根本原因可能是相同的。
除去Spark_*_2.12中的传递的avro映射依赖,对我来说有魔力。现在,我要执行之前失败的Gradle构建:
找不到avro 1.11.0-hadoop2.jar(org.apache.avro:avro:1.11.0)。
但仍不清楚为什么Gradle会以一种奇怪的方式寻找一个完全不同的版本,并将-hadoop2添加到版本中。
https://stackoverflow.com/questions/70281068
复制相似问题