Apache + Spring + Gradle

Question

我有一个在Spring应用程序中运行的Flink应用程序。应用程序无法在Flink集群中作为jar运行。我能够将应用程序作为一个jar运行在我的本地windows计算机(迷你集群)上。

异常堆栈跟踪如下所示：

org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: null
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:546)
        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)
        at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:83)
        at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:80)
        at org.apache.flink.runtime.webmonitor.handlers.utils.JarHandlerUtils$JarHandlerContext.toJobGraph(JarHandlerUtils.java:126)
        at org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$getJobGraphAsync$6(JarRunHandler.java:142)
        at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:50)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:51)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)
        ... 9 more
Caused by: org.apache.flink.client.program.OptimizerPlanEnvironment$ProgramAbortException
        at org.apache.flink.streaming.api.environment.StreamPlanEnvironment.execute(StreamPlanEnvironment.java:70)
        at org.apache.flink.streaming.api.environment.StreamPlanEnvironment.execute(StreamPlanEnvironment.java:53)
        at cdc.il.cdm.application.IngestionJob.execute(IngestionJob.java:61)
        at cdc.il.cdm.application.CdmApplication.run(CdmApplication.java:28)
        at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:813)
        at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:797)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:324)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1260)

我正在使用Gradle和spring gradle-plugin构建工件。似乎构建的工件与Flink的运行时所期望的不匹配。有人有使用Gradle创建工件的好方法吗？

编辑：我找到了一个使用阴影插件- https://imperceptiblethoughts.com/shadow/的解决方案。

下面的gradle构建文件在Flink机器上为我工作。这个罐子是由影子插件建造的。

buildscript {
    repositories {
        jcenter()
        mavenCentral()
    }
    dependencies {
        classpath("org.springframework.boot:spring-boot-gradle-plugin:2.1.6.RELEASE")
        classpath 'com.github.jengelman.gradle.plugins:shadow:5.1.0'
    }
}

apply plugin: 'com.github.johnrengelman.shadow'
apply plugin: 'java'
apply plugin: 'eclipse'
apply plugin: 'idea'
apply plugin: 'org.springframework.boot'
apply plugin: 'io.spring.dependency-management'

ext {
    javaVersion = '1.8'
    flinkVersion = '1.8.0'
    scalaBinaryVersion = '2.11'
    slf4jVersion = '1.7.7'
    log4jVersion = '1.2.17'
}

repositories {
    mavenCentral()
}

sourceCompatibility = 1.8
targetCompatibility = 1.8

dependencies {
    compile('org.springframework.boot:spring-boot-starter-actuator')
    compile('org.springframework.boot:spring-boot-starter-web')
    compile project(':common')
    compile project(':ingestion')
    compile project(':enrichment')
    compile project(':filter')
    compile "org.apache.flink:flink-java:${flinkVersion}"
    compile "org.apache.flink:flink-connector-kafka_2.11:${flinkVersion}"
    compile "org.apache.flink:flink-streaming-java_${scalaBinaryVersion}:${flinkVersion}"

    compile "log4j:log4j:${log4jVersion}"
    compile "org.slf4j:slf4j-log4j12:${slf4jVersion}"
    compile group: 'org.apache.flink', name: 'flink-connector-kafka_2.12', version: '1.8.0'

}

configurations {
    all*.exclude module : 'spring-boot-starter-logging'
}

bootJar {
    enabled false
}

import com.github.jengelman.gradle.plugins.shadow.transformers.*
//def timestamp = new Date().format('yyyy-MM-dd-HHmm')

shadowJar {
    zip64 true
    mergeServiceFiles()
    baseName = 'cdm-flink-app'
    classifier=null
    //version = timestamp
    manifest {
        attributes 'Main-Class': 'cdc.il.cdm.application.CdmApplication'
    }
    append 'META-INF/spring.handlers'
    append 'META-INF/spring.schemas'
    append 'META-INF/spring.tooling'
    transform(PropertiesFileTransformer) {
        paths = ['META-INF/spring.factories' ]
        mergeStrategy = "append"
    }

}

Answer 1

在我看来，您可能没有定义mainClassName，如

mainClassName = 'org.myorg.quickstart.StreamingJob'

查看一下Flink的Gradle快速入门文档，了解更多信息：quickstart.html#gradle。