包含fork/ join和decesion节点的oozie工作流

Question

    <?xml version="1.0" encoding="UTF-8"?>
    <workflow-app xmlns="uri:oozie:workflow:0.5" name="shell-wf">


       <start to="fork_node" />

       <fork name = "fork_node">
          <path start = "query1"/>
          <path start = "query2"/>
       </fork>

       <action name="query1">
          <shell xmlns="uri:oozie:shell-action:0.1">
             <job-tracker>${jobTracker}</job-tracker>
             <name-node>${nameNode}</name-node>
             <configuration>
                <property>
                   <name>mapred.job.queue.name</name>
                   <value>${queueName}</value>
                </property>
             </configuration>
             <exec>oozie.sh</exec>
             <file>oozie.sh#oozie.sh</file>
             <capture-output />
          </shell>
          <ok to="join_node" />
          <error to="postgre"/>
       </action>

       <action name="query2">
          <shell xmlns="uri:oozie:shell-action:0.1">
             <job-tracker>${jobTracker}</job-tracker>
             <name-node>${nameNode}</name-node>
             <configuration>
                <property>
                   <name>mapred.job.queue.name</name>
                   <value>${queueName}</value>
                </property>
             </configuration>
             <exec>oozie2.sh</exec>
             <file>oozie2.sh#oozie2.sh</file>
             <capture-output />
          </shell>
          <ok to="join_node" />
          <error to="postgre" />
       </action>

       <join name = "join_node" to = "postgre"/>

          <action name="postgre">
          <shell xmlns="uri:oozie:shell-action:0.1">
             <job-tracker>${jobTracker}</job-tracker>
             <name-node>${nameNode}</name-node>
             <configuration>
                <property>
                   <name>mapred.job.queue.name</name>
                   <value>${queueName}</value>
                </property>
             </configuration>
             <exec>oozie3.sh</exec>
             <file>oozie3.sh#oozie3.sh</file>
             <capture-output />
          </shell>
          <ok to="decesion-node" />
          <error to="fail" />
       </action>

       <decision name="decesion-node">
            <switch>
                <case to="end">
                  ${wf:lastErrorNode()} eq ""
                </case>
                <default to="fail"/>
            </switch>
        </decision>


        <kill name="fail">
          <message>Action failed, error message[${wf:errorMessage(wf:lastErrorNode())}  ]</message>
       </kill>
       <end name="end" />
    </workflow-app>

尝试运行上述workflow.xml时出现以下错误:从节点决策节点到节点末端的转换无效--在Fork/Join中不允许类型为' end‘的节点

我的用例是总是转换到postgre操作，而不管作业是被终止还是成功，一旦postre操作被执行，如果任何操作失败，它应该去kill node，否则到end node。

Answer 1

当你在fork join中运行2个MR代码时，它需要你集群中的MR的可用性。所以当你运行超过2个MR代码时，它可能会在fork和join中失败。你可以分享你的外壳文件，或者你可以检查你是否正在从u .sh文件中执行正常的*.sh命令，或者从shell文件中执行任何MR代码。

因此，如果您正在运行MR代码，请尝试在您的集群中增加MR

谢谢，学习愉快:D