StreamSets处理器找不到红宝石库

Question

我有两个装有Redis和StreamSets的容器。我想用Java编写自定义处理器，然后放到管道中。但是，当我将代码从教程添加到处理器，将jar发送到lib并尝试启动时，我得到了一个异常。你能帮帮我吗？

处理器教程- https://github.com/streamsets/tutorials/blob/master/tutorial-processor/readme.md

Redis教程- https://dzone.com/articles/quickstart-how-to-use-redis-on-java

  redis:
    image: redis
    ports:
      - 6379:6379
    command: redis-server


  streamsets:
    build:
      context: "."
      args:
        SDC_LIBS: streamsets-datacollector-apache-kafka_2_0-lib                ports:
      - 18630:18630
    volumes:
      - ./user-libs:/opt/streamsets-datacollector-3.9.0/user-libs

在样本处理器中启动：

  @Override
  protected List<ConfigIssue> init() {
      ....
      Config config = new Config();
      config.useSingleServer().setAddress("redis://redis:6379");
      redisson = Redisson.create(config);
      ... }



  <dependency>
      <groupId>org.redisson</groupId>
      <artifactId>redisson</artifactId>
      <version>3.10.4</version>
      <scope>provided</scope>
  </dependency>

对处理器代码的测试工作成功。

日志

streamsets_1  | 2020-03-21 14:00:19,954 [user:] [pipeline:] [runner:] [thread:main] [stage:] ERROR Main - Abnormal exit: java.lang.NoClassDefFoundError: Lorg/redisson/api/RedissonClient;

streamsets_1  | java.lang.NoClassDefFoundError: Lorg/redisson/api/RedissonClient;
streamsets_1  |         at java.lang.Class.getDeclaredFields0(Native Method)
streamsets_1  |         at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
streamsets_1  |         at java.lang.Class.privateGetPublicFields(Class.java:2614)
streamsets_1  |         at java.lang.Class.privateGetPublicFields(Class.java:2629)
streamsets_1  |         at java.lang.Class.getFields(Class.java:1557)
streamsets_1  |         at com.streamsets.datacollector.definition.ConfigDefinitionExtractor.validate(ConfigDefinitionExtractor.java:82)
streamsets_1  |         at com.streamsets.datacollector.definition.ConfigDefinitionExtractor.validate(ConfigDefinitionExtractor.java:65)
streamsets_1  |         at com.streamsets.datacollector.definition.StageDefinitionExtractor.validate(StageDefinitionExtractor.java:141)
streamsets_1  |         at com.streamsets.datacollector.definition.StageDefinitionExtractor.extract(StageDefinitionExtractor.java:197)
streamsets_1  |         at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.loadStages(ClassLoaderStageLibraryTask.java:557)
streamsets_1  |         at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.initTask(ClassLoaderStageLibraryTask.java:278)
streamsets_1  |         at com.streamsets.datacollector.task.AbstractTask.init(AbstractTask.java:62)
streamsets_1  |         at com.streamsets.datacollector.task.CompositeTask.initTask(CompositeTask.java:44)
streamsets_1  |         at com.streamsets.datacollector.task.AbstractTask.init(AbstractTask.java:62)
streamsets_1  |         at com.streamsets.datacollector.task.TaskWrapper.init(TaskWrapper.java:40)
streamsets_1  |         at com.streamsets.datacollector.main.Main.lambda$doMain$1(Main.java:123)
streamsets_1  |         at java.security.AccessController.doPrivileged(Native Method)
streamsets_1  |         at com.streamsets.datacollector.security.SecurityUtil.doAs(SecurityUtil.java:92)
streamsets_1  |         at com.streamsets.datacollector.main.Main.doMain(Main.java:158)
streamsets_1  |         at com.streamsets.datacollector.main.DataCollectorMain.main(DataCollectorMain.java:53)
streamsets_1  |         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
streamsets_1  |         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
streamsets_1  |         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
streamsets_1  |         at java.lang.reflect.Method.invoke(Method.java:498)
streamsets_1  |         at com.streamsets.pipeline.BootstrapMain.bootstrap(BootstrapMain.java:351)
streamsets_1  |         at com.streamsets.pipeline.BootstrapMain.main(BootstrapMain.java:110)
streamsets_1  | Caused by: java.lang.ClassNotFoundException: org.redisson.api.RedissonClient
streamsets_1  |         at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
streamsets_1  |         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
streamsets_1  |         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
streamsets_1  |         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
streamsets_1  |         at com.streamsets.pipeline.SDCClassLoader.loadClass(SDCClassLoader.java:353)
streamsets_1  |         at com.streamsets.pipeline.SDCClassLoader.loadClass(SDCClassLoader.java:316)
streamsets_1  |         at com.streamsets.pipeline.SDCClassLoader.loadClass(SDCClassLoader.java:353)
streamsets_1  |         at com.streamsets.pipeline.SDCClassLoader.loadClass(SDCClassLoader.java:316)
streamsets_1  |         ... 26 more
streamsets_1  |
streamsets_1  | Abnormal exit: java.lang.NoClassDefFoundError: Lorg/redisson/api/RedissonClient;Check STDERR for more details
streamsets_1  |
streamsets_1  | java.lang.NoClassDefFoundError: Lorg/redisson/api/RedissonClient;
streamsets_1  |         at java.lang.Class.getDeclaredFields0(Native Method)
streamsets_1  |         at java.lang.Class.privateGetDeclaredFields(Class.java:2583)
streamsets_1  |         at java.lang.Class.privateGetPublicFields(Class.java:2614)
streamsets_1  |         at java.lang.Class.privateGetPublicFields(Class.java:2629)
streamsets_1  |         at java.lang.Class.getFields(Class.java:1557)
streamsets_1  |         at com.streamsets.datacollector.definition.ConfigDefinitionExtractor.validate(ConfigDefinitionExtractor.java:82)
streamsets_1  |         at com.streamsets.datacollector.definition.ConfigDefinitionExtractor.validate(ConfigDefinitionExtractor.java:65)
streamsets_1  |         at com.streamsets.datacollector.definition.StageDefinitionExtractor.validate(StageDefinitionExtractor.java:141)
streamsets_1  |         at com.streamsets.datacollector.definition.StageDefinitionExtractor.extract(StageDefinitionExtractor.java:197)
streamsets_1  |         at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.loadStages(ClassLoaderStageLibraryTask.java:557)
streamsets_1  |         at com.streamsets.datacollector.stagelibrary.ClassLoaderStageLibraryTask.initTask(ClassLoaderStageLibraryTask.java:278)
streamsets_1  |         at com.streamsets.datacollector.task.AbstractTask.init(AbstractTask.java:62)
streamsets_1  |         at com.streamsets.datacollector.task.CompositeTask.initTask(CompositeTask.java:44)
streamsets_1  |         at com.streamsets.datacollector.task.AbstractTask.init(AbstractTask.java:62)
streamsets_1  |         at com.streamsets.datacollector.task.TaskWrapper.init(TaskWrapper.java:40)
streamsets_1  |         at com.streamsets.datacollector.main.Main.lambda$doMain$1(Main.java:123)
streamsets_1  |         at java.security.AccessController.doPrivileged(Native Method)
streamsets_1  |         at com.streamsets.datacollector.security.SecurityUtil.doAs(SecurityUtil.java:92)
streamsets_1  |         at com.streamsets.datacollector.main.Main.doMain(Main.java:158)
streamsets_1  |         at com.streamsets.datacollector.main.DataCollectorMain.main(DataCollectorMain.java:53)
streamsets_1  |         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
streamsets_1  |         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
streamsets_1  |         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
streamsets_1  |         at java.lang.reflect.Method.invoke(Method.java:498)
streamsets_1  |         at com.streamsets.pipeline.BootstrapMain.bootstrap(BootstrapMain.java:351)
streamsets_1  |         at com.streamsets.pipeline.BootstrapMain.main(BootstrapMain.java:110)
streamsets_1  | Caused by: java.lang.ClassNotFoundException: org.redisson.api.RedissonClient
streamsets_1  |         at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
streamsets_1  |         at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
streamsets_1  |         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
streamsets_1  |         at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
streamsets_1  |         at com.streamsets.pipeline.SDCClassLoader.loadClass(SDCClassLoader.java:353)
streamsets_1  |         at com.streamsets.pipeline.SDCClassLoader.loadClass(SDCClassLoader.java:316)
streamsets_1  |         at com.streamsets.pipeline.SDCClassLoader.loadClass(SDCClassLoader.java:353)
streamsets_1  |         at com.streamsets.pipeline.SDCClassLoader.loadClass(SDCClassLoader.java:316)
streamsets_1  |         ... 26 more

Answer 1

问题很可能是你的依赖范围。您已经指定了provided，它被定义为：

这非常类似于compile，但表示您希望JDK或容器在运行时提供依赖关系。例如，在为构建web应用程序时，您将将对Servlet和相关Java的依赖设置为provided范围，因为web容器提供了这些类。此范围仅在编译和测试类路径上可用，且不可传递。

您应该指定compile作用域。