Rhadoop hdfs.init()错误

Question

我最近安装了CDH5.1.0和R3.1.*，并且我已经正确安装了rmr2、rJava和rhdfs。(以及所需的包和设置所需的环境变量)在安装rhdfs遇到一些问题后，我将其添加到我的/usr/lib/R/etc/Renvi.site文件中：

HADOOP_HOME="usr/lib/hadoop"
HADOOP_CMD="usr/bin/hadoop"
HADOOP_STREAMING="usr/lib/hadoop-mapreduce/hadoop-streaming-2.3.0-cdh5.1.0.jar"

然后启动R并运行以下代码：

>library(rmr2)
 loading required packages ...
>library(rJava)
>library(rhdfs)

HADOOP_CMD=usr/bin/hadoop

be sure to run hdfs.init()
>hdfs.init()
sh: 1: usr/bin/hadoop: not found
Error in system(command, intern = TRUE) : error in running command

我在java类路径上也看到过类似的问题，但是我在互联网上没有发现这个特定的问题！任何帮助都将不胜感激。

Answer 1

我也有同样的问题，HDP 2.1。看看从MapR链接和RevR on GitHub。似乎需要将LD_LIBRARY_PATH变量设置为rJava.so变量。

HADOOP_STREAMING="/usr/lib/hadoop-mapreduce/hadoop-streaming-2.2.0.2.0.6.0-76.jar"
HADOOP_CONF="/etc/hadoop/conf"
LD_LIBRARY_PATH="/usr/lib64/R/library/rJava/libs/rJava.so"
HADOOP_COMMON_LIB_NATIVE_DIR="/usr/lib/hadoop/lib/native/"

现在，当在R中运行rhdfs时，我会收到以下警告，但它似乎有效：

> hdfs.init()
14/11/12 09:20:42 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
14/11/12 09:20:43 WARN hdfs.BlockReaderLocal: The short-circuit local reads feature cannot be used because libhadoop cannot be loaded.

> hdfs.ls(".")
  permission owner group    size          modtime                  file
1 drwx------  root  root       0 2014-11-07 09:50   /user/root/.staging