齐柏林飞艇配Spark2

Question

我正在尝试配置齐柏林飞艇，以便在cloudera中使用Spark2和cloudera版本5.10.1。

​

我得到的是错误"org.apache.zeppelin.interpreter.InterpreterException:opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/bin/spark2-submit/bin/spark-submit:而不是目录“

很明显，它会将“/bin/火花-提交”附加到路径中。我该怎么纠正呢？

Answer 1

将"/opt/cloudera/parcels/SPARK2-2.1.0.cloudera1-1.cdh5.7.0.p0.120904/“设置为SPARK_HOME可能还不够。在我的例子中，spark2开始在齐柏林飞艇中工作，当我将SPARK_HOME设置为：

SPARK_HOME=/opt/cloudera/parcels/SPARK2/lib/spark2

*SPARK2 2与较长的包裹名称相连。

进一步发展我的答案。“莫洛托”的解决方案让我在齐柏林飞艇上犯了这样的错误：

org.apache.zeppelin.interpreter.InterpreterException: /opt/cloudera/parcels/SPARK2/bin/spark-submit: line 17: //../../CDH/lib/bigtop-utils/bigtop-detect-javahome: No such file or directory
/opt/cloudera/parcels/SPARK2/bin/spark-submit: line 19: //../lib/spark2/bin/spark-submit: No such file or directory

这可能与“火花2-提交”如何试图定位它需要工作的路径有关。主要是：

#!/bin/bash
  # Reference: http://stackoverflow.com/questions/59895/can-a-bash-script-tell-what-directory-its-stored-in
  SOURCE="${BASH_SOURCE[0]}"
  BIN_DIR="$( dirname "$SOURCE" )"
  while [ -h "$SOURCE" ]
  do
    SOURCE="$(readlink "$SOURCE")"
    [[ $SOURCE != /* ]] && SOURCE="$DIR/$SOURCE"
    BIN_DIR="$( cd -P "$( dirname "$SOURCE"  )" && pwd )"
  done
  BIN_DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"
  CDH_LIB_DIR=$BIN_DIR/../../CDH/lib
  LIB_DIR=$BIN_DIR/../lib
export HADOOP_HOME=$CDH_LIB_DIR/hadoop

希望这能帮到别人。:)