KafkaEmbedded为SecurityProtocol抛出ClassNotFoundException

Question

我的生产代码运行得很好，但是我的集成测试抛出了SecurityProtocol的ClassNotFoundException。我使用的是spring-boot-parent 1.5.10.RELEASE。

    <dependency>
        <groupId>org.springframework.kafka</groupId>
        <artifactId>spring-kafka</artifactId>
        <version>1.3.3.RELEASE</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.kafka</groupId>
        <artifactId>spring-kafka-test</artifactId>
        <version>1.3.3.RELEASE</version>
        <scope>test</scope>
    </dependency>

我得到了以下异常：

java.lang.NoClassDefFoundError: org/apache/kafka/common/security/auth/SecurityProtocol

    at kafka.server.KafkaConfig$.<init>(KafkaConfig.scala:578)
    at kafka.server.KafkaConfig$.<clinit>(KafkaConfig.scala)
    at kafka.utils.TestUtils$.createBrokerConfig(TestUtils.scala:228)
    at kafka.utils.TestUtils.createBrokerConfig(TestUtils.scala)
    at org.springframework.kafka.test.rule.KafkaEmbedded.createProperties(KafkaEmbedded.java:214)
    at org.springframework.kafka.test.rule.KafkaEmbedded.before(KafkaEmbedded.java:191)
    at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
    at org.junit.rules.RunRules.evaluate(RunRules.java:20)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.ClassNotFoundException: org.apache.kafka.common.security.auth.SecurityProtocol
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 15 more

当我使用这个pom时，我得到了一个不同的错误：

<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka-test</artifactId>
    <scope>test</scope>
</dependency>

java.lang.NoClassDefFoundError: kafka/utils/Time
    at com.betstars.eventdata.app.kafka.message.KafkaMessageSenderTest.<clinit>(KafkaMessageSenderTest.java:39)
    at sun.misc.Unsafe.ensureClassInitialized(Native Method)
    at sun.reflect.UnsafeFieldAccessorFactory.newFieldAccessor(UnsafeFieldAccessorFactory.java:43)
    at sun.reflect.ReflectionFactory.newFieldAccessor(ReflectionFactory.java:156)
    at java.lang.reflect.Field.acquireFieldAccessor(Field.java:1088)
    at java.lang.reflect.Field.getFieldAccessor(Field.java:1069)
    at java.lang.reflect.Field.get(Field.java:393)
    at org.junit.runners.model.FrameworkField.get(FrameworkField.java:73)
    at org.junit.runners.model.TestClass.getAnnotatedFieldValues(TestClass.java:230)
    at org.junit.runners.ParentRunner.classRules(ParentRunner.java:255)
    at org.junit.runners.ParentRunner.withClassRules(ParentRunner.java:244)
    at org.junit.runners.ParentRunner.classBlock(ParentRunner.java:194)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:362)
    at org.springframework.test.context.junit4.SpringJUnit4ClassRunner.run(SpringJUnit4ClassRunner.java:191)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: java.lang.ClassNotFoundException: kafka.utils.Time
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    ... 19 more

当我使用最新版本时，我得到了一个不同的错误：

<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka</artifactId>
    <version>2.1.2.RELEASE</version>
</dependency>
<dependency>
    <groupId>org.springframework.kafka</groupId>
    <artifactId>spring-kafka-test</artifactId>
    <version>2.1.2.RELEASE</version>
    <scope>test</scope>
</dependency>

java.lang.NoSuchMethodError: org.springframework.util.Assert.state(ZLjava/util/function/Supplier;)V

    at org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.determineInferredType(MessagingMessageListenerAdapter.java:396)
    at org.springframework.kafka.listener.adapter.MessagingMessageListenerAdapter.<init>(MessagingMessageListenerAdapter.java:100)
    at org.springframework.kafka.listener.adapter.RecordMessagingMessageListenerAdapter.<init>(RecordMessagingMessageListenerAdapter.java:61)
    at org.springframework.kafka.config.MethodKafkaListenerEndpoint.createMessageListenerInstance(MethodKafkaListenerEndpoint.java:172)
    at org.springframework.kafka.config.MethodKafkaListenerEndpoint.createMessageListener(MethodKafkaListenerEndpoint.java:132)
    at org.springframework.kafka.config.AbstractKafkaListenerEndpoint.setupMessageListener(AbstractKafkaListenerEndpoint.java:355)
    at org.springframework.kafka.config.AbstractKafkaListenerEndpoint.setupListenerContainer(AbstractKafkaListenerEndpoint.java:340)

Answer 1

org/apache/kafka/common/security/auth/SecurityProtocol：

java.lang.NoClassDefFoundError

这是由不匹配的kafka_2.11和kafka-clients jars引起的；它们必须是相同的版本。在1.0.0 kafka_clients jar中，SecurityProtocol被移动到不同的包中。

org.springframework.util.Assert.state(ZLjava/util/function/Supplier;)V：

java.lang.NoSuchMethodError

这是因为spring-kafka 2.x需要Spring Framework5.0，而Spring Framework5.0与boot1.5不兼容；它使用Spring Framework4.3.x。

Spring Boot 2.0即将发布，它将与最新的Spring Kafka一起工作。

java.lang.NoClassDefFoundError: kafka/utils/Time

这是另一个jar不匹配

KAFKA-2247; Merge kafka.utils.Time and kafka.common.utils.Time

由于boot做依赖管理的方式，当你覆盖spring-kafka的版本时，你也必须覆盖对正确版本的依赖(kafka客户端，kafka_2.11)。

Answer 2

令人惊讶的是，我的服务器正在运行，因此我的集成测试失败了。我停止了正在运行的服务器并启动了测试，它工作得很好，没有任何明确的版本定义。

Answer 3

作为对可接受答案的补充，我将描述我的解决方案，因为在我的例子中，没有找到的类是protocol/SecurityProtocol

java.lang.NoClassDefFoundError: org/apache/kafka/common/protocol/SecurityProtocol

security/auth/SecurityProtocol的哪一项不同

java.lang.NoClassDefFoundError: org/apache/kafka/common/security/auth/SecurityProtocol

基本上，我必须使用./mvnw dependency:tree检查所有依赖项，并排除不在项目版本上的kafka-clients。

<exclusions>
  <exclusion>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka_2.11</artifactId>
  </exclusion>
  <exclusion>
    <groupId>org.apache.kafka</groupId>
    <artifactId>kafka-clients</artifactId>
  </exclusion>
</exclusions>

其中大部分是3.0版。在Apache Kafka repo上有一个提示，它必须与scala版本兼容。然后，还有必要验证项目中的所有scala版本。我有几个在使用不同的版本。

Scala2.12支持从Apache Kafka3.0开始就被弃用了

并添加唯一具有org/apache/kafka/common/protocol/SecurityProtocol类的版本，它与Scala2.11兼容。

<dependency>
  <groupId>org.apache.kafka</groupId>
  <artifactId>kafka-clients</artifactId>
  <version>0.11.0.0</version>
</dependency>