Spring Cloud Dataflow重复键错误(此作业的作业执行已在运行)

Question

我有一个Spring Cloud数据流，其中包含Http源、自定义处理器和任务启动器

​

我有一个从stream - task-launcher接收器调用的组合任务。

​

​

当只执行一个实例时，组合任务可以正确启动并成功运行，但当创建多个实例时，它们会间歇性地出现重复键错误。我最近更新到了以下版本：

• bitnami/spring-cloud-dataflow 
version: 2.9.0-debian-10-r15
• bitnami/spring-cloud-dataflow-composed-task-runner
version: 2.9.0-debian-10-r17
• bitnami/spring-cloud-skipper
version: 2.8.0-debian-10-r15

java.lang.IllegalStateException: Failed to execute ApplicationRunner
2021-10-28T16:53:59.179909948Z  at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:785) [spring-boot-2.5.5.jar!/:2.5.5]
2021-10-28T16:53:59.179913386Z  at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:772) [spring-boot-2.5.5.jar!/:2.5.5]
2021-10-28T16:53:59.179916469Z  at org.springframework.boot.SpringApplication.run(SpringApplication.java:345) [spring-boot-2.5.5.jar!/:2.5.5]
2021-10-28T16:53:59.179919669Z  at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343) [spring-boot-2.5.5.jar!/:2.5.5]
2021-10-28T16:53:59.179922662Z  at org.springframework.boot.SpringApplication.run(SpringApplication.java:1332) [spring-boot-2.5.5.jar!/:2.5.5]
2021-10-28T16:53:59.179925769Z  at org.springframework.cloud.dataflow.composedtaskrunner.ComposedTaskRunner.main(ComposedTaskRunner.java:31) [classes!/:2.9.0]
2021-10-28T16:53:59.179928817Z  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_312]
2021-10-28T16:53:59.179931884Z  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_312]
2021-10-28T16:53:59.179934872Z  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_312]
2021-10-28T16:53:59.179937847Z  at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_312]
2021-10-28T16:53:59.179940922Z  at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49) [spring-cloud-dataflow-composed-task-runner-2.9.0.jar:2.9.0]
2021-10-28T16:53:59.179944118Z  at org.springframework.boot.loader.Launcher.launch(Launcher.java:108) [spring-cloud-dataflow-composed-task-runner-2.9.0.jar:2.9.0]
2021-10-28T16:53:59.179947158Z  at org.springframework.boot.loader.Launcher.launch(Launcher.java:58) [spring-cloud-dataflow-composed-task-runner-2.9.0.jar:2.9.0]
2021-10-28T16:53:59.179950209Z  at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88) [spring-cloud-dataflow-composed-task-runner-2.9.0.jar:2.9.0]
2021-10-28T16:53:59.179953277Z Caused by: org.springframework.batch.core.repository.JobExecutionAlreadyRunningException: A job execution for this job is already running: JobInstance: id=1200, version=0, Job=[composedtask]
2021-10-28T16:53:59.179957212Z  at org.springframework.batch.core.repository.support.SimpleJobRepository.createJobExecution(SimpleJobRepository.java:127) ~[spring-batch-core-4.3.3.jar!/:4.3.3]
2021-10-28T16:53:59.179960544Z  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_312]
2021-10-28T16:53:59.179963557Z  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_312]
2021-10-28T16:53:59.179975310Z  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_312]
2021-10-28T16:53:59.179978588Z  at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_312]
2021-10-28T16:53:59.179981557Z  at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.179984657Z  at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.179987854Z  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.179990997Z  at org.springframework.transaction.interceptor.TransactionInterceptor$1.proceedWithInvocation(TransactionInterceptor.java:123) ~[spring-tx-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.179994173Z  at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:388) ~[spring-tx-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.179997313Z  at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:119) ~[spring-tx-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.180000409Z  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.180003545Z  at org.springframework.batch.core.repository.support.AbstractJobRepositoryFactoryBean$1.invoke(AbstractJobRepositoryFactoryBean.java:181) ~[spring-batch-core-4.3.3.jar!/:4.3.3]
2021-10-28T16:53:59.180006834Z  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.180009937Z  at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.180013021Z  at com.sun.proxy.$Proxy64.createJobExecution(Unknown Source) ~[na:na]
2021-10-28T16:53:59.180016256Z  at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:137) ~[spring-batch-core-4.3.3.jar!/:4.3.3]
2021-10-28T16:53:59.180019375Z  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_312]
2021-10-28T16:53:59.180022367Z  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_312]
2021-10-28T16:53:59.180025349Z  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_312]
2021-10-28T16:53:59.180028473Z  at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_312]
2021-10-28T16:53:59.180031405Z  at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.180034563Z  at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.180037663Z  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.180042900Z  at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:128) ~[spring-batch-core-4.3.3.jar!/:4.3.3]
2021-10-28T16:53:59.180046230Z  at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.180049362Z  at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:215) ~[spring-aop-5.3.10.jar!/:5.3.10]
2021-10-28T16:53:59.180052538Z  at com.sun.proxy.$Proxy67.run(Unknown Source) ~[na:na]
2021-10-28T16:53:59.180055536Z  at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:199) ~[spring-boot-autoconfigure-2.5.5.jar!/:2.5.5]
2021-10-28T16:53:59.180058822Z  at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:173) ~[spring-boot-autoconfigure-2.5.5.jar!/:2.5.5]
2021-10-28T16:53:59.180061980Z  at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:160) ~[spring-boot-autoconfigure-2.5.5.jar!/:2.5.5]
2021-10-28T16:53:59.180065453Z  at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:155) ~[spring-boot-autoconfigure-2.5.5.jar!/:2.5.5]
2021-10-28T16:53:59.180068728Z  at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:150) ~[spring-boot-autoconfigure-2.5.5.jar!/:2.5.5]
2021-10-28T16:53:59.180071912Z  at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:782) [spring-boot-2.5.5.jar!/:2.5.5]

几个月前，当我们运行scdf 2.8.1时，我报告了这个问题，我被告知这个问题已经得到承认，并将在未来的版本中得到纠正。有关于这个问题的任何更新或解决方法吗？

我尝试在任务启动器接收器上使用--increment instance-enabled=true，但这没有帮助。

谢谢!

Answer 1

不幸的是，我们似乎没有将错误修复(即：4684)移植到2.8.x分支，所以这就是它还不可用的原因。

尽管如此，这个修复包含在最新的2.9.0/2.9.1版本中-请尝试一下，如果发现异常请告诉我们。

Answer 2

Spring Cloud Dataflow升级到2.9.0后

在VMWare的帮助下，我终于弄明白了这一点。答案是设置启用uuid实例的property=true。

最初，我试图在任务启动器的流部署属性上设置它，但这是不正确的。

在创建任务启动请求消息的自定义处理器中，我设置了uuid-instance-enabled=true属性，这更正了这个问题。这里设置的属性将过滤到组合任务运行器(kubernetes pod)，该运行器被启动以执行每个组合任务。

示例代码：

List<String> composedTaskArgs = new ArrayList<>();
composedTaskArgs.add("--dataflow-server-uri="+scdfProperties.getDataflowServerUri());
composedTaskArgs.add("--uuid-instance-enabled=true");

json.put("args", composedTaskArgs);
json.put("deploymentProps", props);

message = json.toString();

logger.info("Send to tasklauncher: "+message);
return new GenericMessage<>(message);