ImportError:无法从“”graphframes.lib“”导入名称“”Pregel“”

Question

我使用的是jupyter的pyspark和graphframes。我能够成功地导入pyspark和graphframes，但当我尝试时：

from graphframes.lib import Pregel 

我得到以下错误：

ImportError: cannot import name 'Pregel' from 'graphframes.lib'

这篇文章是我如何在没有graphframes.lib的情况下让graphframes工作的：

https://github.com/graphframes/graphframes/issues/104

wget https://github.com/graphframes/graphframes/archive/release-0.2.0.zip
unzip release-0.2.0.zip
cd graphframes-release-0.2.0
build/sbt assembly
cd ..

# Copy necessary files to root level so we can start pyspark. 
cp graphframes-release-0.2.0/target/scala-2.11/graphframes-release-0-2-0-assembly-0.2.0-spark2.0.jar .
cp -r graphframes-release-0.2.0/python/graphframes .

# Set environment to use Jupyter
export PYSPARK_DRIVER_PYTHON=jupyter
export PYSPARK_DRIVER_PYTHON_OPTS=notebook

# Launch the jupyter server.
pyspark --jars graphframes-release-0-2-0-assembly-0.2.0-spark2.0.jar

我尝试重复上面的命令，没有环境行，因为pyspark在jupyter中对我来说工作得很好，使用了不同的版本，并且能够获得graphframes.lib，但没有Pregel：

wget https://github.com/graphframes/graphframes/archive/release-0.6.0.zip
unzip release-0.6.0.zip
cd graphframes-release-0.6.0
build/sbt assembly
cd ..

# Copy necessary files to root level so we can start pyspark. 
cp graphframes-release-0.6.0/target/scala-2.11/graphframes-assembly-0.6.0-spark2.3.jar .
cp -r graphframes-release-0.6.0/python/graphframes .

# Set environment to use Jupyter
export PYSPARK_DRIVER_PYTHON=jupyter
export PYSPARK_DRIVER_PYTHON_OPTS=notebook

# Launch the jupyter server.
pyspark --jars graphframes-assembly-0.6.0-spark2.3.jar

现在我可以看到graphrames.lib目录，但里面只有aggregate_messages.py。

最后，我尝试了以下方法，但得到了一个404错误：

wget https://github.com/graphframes/graphframes/archive/release-0.7.0.zip

我希望，因为我能够导入字框，所以我能够从graphframes.lib导入Pregel。在我现在的0.6.0版本中，似乎有一个graphrames.lib，但是没有Pregel，而且还没有针对graphframes的0.7.0版本。

Answer 1

我使用以下方法解决了这个错误：

wget https://github.com/graphframes/graphframes/archive/f9e13ab4ac1a7113f8439744a1ab45710eb50a72.zip
unzip graphframes-f9e13ab4ac1a7113f8439744a1ab45710eb50a72.zip
cd graphframes-f9e13ab4ac1a7113f8439744a1ab45710eb50a72
build/sbt assembly
cd ..

# Copy necessary files to root level so we can start pyspark. 
cp graphframes-f9e13ab4ac1a7113f8439744a1ab45710eb50a72/target/scala-2.11/graphframes-assembly-0.7.0-spark2.4.jar .
cp -r graphframes-f9e13ab4ac1a7113f8439744a1ab45710eb50a72/python/graphframes .

# Set environment to use Jupyter (if jupyter working with pyspark, skip)
# export PYSPARK_DRIVER_PYTHON=jupyter
# export PYSPARK_DRIVER_PYTHON_OPTS=notebook

# launch pyspark
pyspark --jars graphframes-assembly-0.7.0-spark2.4.jar