使用oozie执行sqoop作业时出错

Question

当我使用下面的oozie详细信息提交sqoop作业时，我会得到这个错误。你能帮忙吗？

我的workflow.xml文件

<workflow-app xmlns="uri:oozie:workflow:0.4" name="oozie-wf">
<start to="sqoop-wf1"/>
<action name="sqoop-wf1">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<command>import --username hadoop --password bigdata --connect  
jdbc:db2://<host string>:50000/msrc --driver com.ibm.db2.jcc.DB2Driver --  
query  
'SELECT * FROM STUDENT.SALES WHERE (1=0 OR $CONDITIONS)' --target-dir  
/user/${wf:user()}/${examplesRoot}/apps/sqoop -m 1</command>
</sqoop>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Action failed</message>
</kill>
<end name="end"/>
</workflow-app>

我的job.properties文件

nameNode=hdfs://sandbox.hortonworks.com:8020
jobTracker=sandbox.hortonworks.com:8050
queueName=default
examplesRoot=examples
oozie.use.system.libpath=true
oozie.libpath=/user/oozie/share/lib
oozie.wf.application.path=${nameNode}/user/root/${examplesRoot}/apps/sqoop

现在调用Sqoop命令行>>>

3638主警告org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR尚未在环境中设置。无法检查其他配置。2015-10-10 05:54:21,293警告主要tool.SqoopTool (SqoopTool.java:loadPluginsFromConfDir(177)) - $SQOOP_CONF_DIR尚未在环境中设置.无法检查其他配置。4540主要信息org.apache.sqoop.Sqoop -运行Sqoop版本: 1.4.6.2.3.0.0-2557 2015-10-1005:54:22,195 INFO主sqoop.Sqoop (Sqoop.java:(92)) -运行Sqoop版本: 1.4.6.2.3.0.0-2557 5581主警告org.apache.sqoop.tool.BaseSqoopTool -在命令行设置密码是不安全的。考虑使用-P代替。2015-10-10 05:54:23,236警告主(BaseSqoopTool.java:applyCredentialsOptions(1021)) -在命令行设置密码是不安全的.考虑使用-P代替。

公认的论点: 15582主要错误org.apache.sqoop.tool.BaseSqoopTool -导入的错误解析参数: 2015-10-10 05:54:23,237错误主tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(303)) -用于导入的错误解析参数: 5582主要错误org.apache.sqoop.tool.BaseSqoopTool -未识别的参数：*2015-10-1005:54:23,237错误主tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(306)) -未识别的参数：* 5583主错误org.apache.sqoop.tool.BaseSqoopTool-不被承认的论点:从2015年开始:10-1005:54:23,238错误主tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(306)) -不被识别的论点:从5583主要错误org.apache.sqoop.tool.BaseSqoopTool -不被识别的论点: STUDENT.SALES 2015-10-1005:54:23,238错误主tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(306)) -未被承认的论点: STUDENT.SALES 5583主要错误org.apache.sqoop.tool.BaseSqoopTool -不被识别的论点:2015年-10-10 05:54:23,238错误主要工具.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(306)) -未被识别的论点:其中5583个主要错误org.apache.sqoop.tool.BaseSqoopTool -未被识别的论点：(1=0 2015-10-1005:54:23,238错误主tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(306)-不被承认的论点：(1=0 5583主要错误org.apache.sqoop.tool.BaseSqoopTool -不被识别的论点:或2015-10-1005:54:23,238错误主tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(306)) -未识别的论点:或5583主要错误org.apache.sqoop.tool.BaseSqoopTool -未识别参数：$CONDITIONS)‘2015-10-1005:54:23,238错误主tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(306)) -未识别参数：$CONDITIONS)’5583主要错误org.apache.sqoop.tool.BaseSqoopTool -未识别参数：-目标-dir 2015-10-10 05:54:23,238错误主tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(306)) -不识别参数：-目标-dir 5584主要错误org.apache.sqoop.tool.BaseSqoopTool -未识别参数:/user/root/示例/app/sqoop 2015-10-1005:54:23,239错误主tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(306)) -未识别的参数:/user/root/示例/app/sqoop 5584主要错误org.apache.sqoop.tool.BaseSqoopTool -未识别的参数：-m 2015-10-10 05:54:23,239错误主tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(306)) -未识别的参数：-m 5584主要错误org.apache.sqoop.tool.BaseSqoopTool - Un 2015-10-1005:54:23,239错误主tool.BaseSqoopTool (BaseSqoopTool.java:hasUnrecognizedArgs(306)) -未被识别的论点:1拦截System.exit(1)

主类的<<<调用已完成<<<

失败的Oozie发射器，主类org.apache.oozie.action.hadoop.SqoopMain，退出代码1

Oozie发射器失败，优雅地完成了Hadoop的工作

将动作数据上传到HDFS序列文件: hdfs://sandbox.hortonworks.com:8020/user/root/oozie-root/0000035-151009124402969-oozie-root-W/sqoop-wf1--sqoop/action-data.seq 2015-10-1005:54:23,322 INFO主zlib.ZlibFactory (ZlibFactory.java:(49)) -成功加载和初始化本机-zlib库2015 10-10 05:54:23 323 INFO main compress.CodecPool (CodecPool.java:getCompressor(153)) - Got全新压缩机.deflate

Oozie发射器结束

2015-10-10 05:54:23,346 INFO主mapred.Task (Task.java:done(1038)) - Task:attempt_1444199299654_0255_m_000000_0完成。并且正在提交2015年-10-10 05:54:23,426 INFO main mapred.Task (Task.java:commit(1199)) -任务attempt_1444199299654_0255_m_000000_0现在允许提交-10-10 05:54:23,438 INFO主output.FileOutputCommitter (FileOutputCommitter.java:commitTask(482)) -保存任务'attempt_1444199299654_0255_m_000000_0‘到Task.java:commit的输出。oozie-root-W/sqoop-wf1--sqoop/output/_temporary/1/task_1444199299654_0255_m_000000 2015-10-1005:54:23,509 INFO主mapred.Task (Task.java:sendDone(1158)) -任务'attempt_1444199299654_0255_m_000000_0‘完成。

Answer 1

当sqoop试图使用Java解析整个命令时，它将使用下面提到的空格进行解析，您可以像下面提到的那样尝试，它将对您很好。

尝尝这个,

<workflow-app xmlns="uri:oozie:workflow:0.4" name="oozie-wf">
<start to="sqoop-wf1"/>
<action name="sqoop-wf1">
<sqoop xmlns="uri:oozie:sqoop-action:0.2">
<job-tracker>${jobTracker}</job-tracker>
<name-node>${nameNode}</name-node>
<arg>import</arg>
<arg>--connect</arg>
<arg>jdbc:db2://<host string>:50000/msrc</arg>
<arg>--username</arg>
<arg>hadoop</arg>
<arg>--password</arg>
<arg>bigdata</arg>
<arg>--driver</arg>
<arg>com.ibm.db2.jcc.DB2Driver</arg>
<arg>--query</arg>
<arg>SELECT * FROM STUDENT.SALES WHERE (1=0 OR $CONDITIONS)</arg>
<arg>--target-dir</arg>
<arg>/user/${wf:user()}/${examplesRoot}/apps/sqoop</arg>
<arg>-m</arg>
<arg>1</arg>
</sqoop>
<ok to="end"/>
<error to="kill"/>
</action>
<kill name="kill">
<message>Action failed</message>
</kill>
<end name="end"/>
</workflow-app>

Answer 2

当使用命令元素时，oozie会将每个空间上的命令拆分成多个arguments.Passing --如果您有一个简单的sql查询，那么arg就很好了，但是如果您有一个具有多个联接的复杂查询，并且传递多个-arg的条件可能很繁琐，那么您应该创建sqoop作业并从Oozie执行它们。

1.配置sqoop-site.xml以使用sqoop亚稳态更改以下属性

sqoop.metastore.client.autoconnect.url,sqoop.metastore.client.autoconnect.username,sqoop.metastore.client.enable.autoconnect,sqoop.metastore.client.autoconnect.password

literal

2.启动sqoop亚稳态

sqoop metastore & 

3.创造sqoop就业机会

sqoop job --create jobid -- import ...

3.在oozie中提交sqoop作业

job --exec jobid --meta-connect sqoop-metastore-URL