首页
学习
活动
专区
圈层
工具
发布
社区首页 >问答首页 >在Hadoop2中以独立模式运行Eclipse reduce作业时出错?

在Hadoop2中以独立模式运行Eclipse reduce作业时出错?
EN

Stack Overflow用户
提问于 2014-07-11 05:15:45
回答 1查看 1K关注 0票数 0

当我在Eclipse中运行MR作业时,我得到了以下错误。

代码语言:javascript
复制
2014-07-10 14:07:30 WARN  NativeCodeLoader:62 - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
job done
2014-07-10 14:07:30 INFO  JvmMetrics:76 - Initializing JVM Metrics with processName=JobTracker, sessionId=
2014-07-10 14:07:30 WARN  JobSubmitter:149 - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2014-07-10 14:07:30 INFO  JobSubmitter:439 - Cleaning up the staging area file:/Users/name/tmp/mapred/staging/sridhar519992773/.staging/job_local519992773_0001
2014-07-10 14:07:30 ERROR UserGroupInformation:1494 - PriviledgedActionException as:sridhar (auth:SIMPLE) cause:org.apache.hadoop.util.Shell$ExitCodeException: chmod: /Users/name/tmp/mapred/staging/sridhar519992773/.staging/job_local519992773_0001: No such file or directory

2014-07-10 14:07:30 ERROR App:43 - Error running MapReduce Job
org.apache.hadoop.util.Shell$ExitCodeException: chmod: /Users/name/tmp/mapred/staging/sridhar519992773/.staging/job_local519992773_0001: No such file or directory

    at org.apache.hadoop.util.Shell.runCommand(Shell.java:464)
    at org.apache.hadoop.util.Shell.run(Shell.java:379)
    at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:589)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:678)
    at org.apache.hadoop.util.Shell.execCommand(Shell.java:661)
    at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:639)
    at org.apache.hadoop.fs.FilterFileSystem.setPermission(FilterFileSystem.java:468)
    at org.apache.hadoop.fs.FileSystem.mkdirs(FileSystem.java:596)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:178)
    at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:300)
    at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:387)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
    at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:394)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
    at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
    at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
    at hadoopstandalone.standalone.App.doMapReduce(App.java:40)
    at hadoopstandalone.standalone.App.main(App.java:27)

这是我的core-site.xml:

代码语言:javascript
复制
<configuration>
<property>
  <name>fs.default.name</name>
  <value>file:///</value>
</property>
<property>
  <name>hadoop.tmp.dir</name>
  <value>/Users/sridhar/Desktop/tmp</value>
</property>
</configuration>

这是我的mapred-site.xml:

代码语言:javascript
复制
<configuration>
<property>
  <name>mapred.job.tracker</name>
  <value>local</value>
</property>
</configuration>
EN

回答 1

Stack Overflow用户

发布于 2014-07-11 15:58:30

core-site.xml应该类似于`

代码语言:javascript
复制
<configuration>
<property>
<name>fs.default.name</name>
<value>hdfs://localhost:9000</value>
</property>

<property>
<name>hadoop.temp.dir</name>
<value>/home/myname/hdfs/temp</value>
</property>
</configuration>

hdfs-site.xml应该如下所示

代码语言:javascript
复制
<configuration>
<property>
<name>dfs.name.dir</name>
<value>/home/myname/hdfs/name</value>
</property>

<property>
<name>dfs.data.dir</name>
<value>/home/myname/hdfs/data</value>
</property>

<property>
<name>dfs.replication</name>
<value>1</value>
</property>

</configuration>

如果有效,请让我知道

票数 0
EN
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接:

https://stackoverflow.com/questions/24686431

复制
相关文章

相似问题

领券
问题归档专栏文章快讯文章归档关键词归档开发者手册归档开发者手册 Section 归档