有没有可能让spark接受本地文件作为输入,但对其进行分布式处理?
我的代码中有sc.textFile(file:///path-to-file-locally),我知道文件的确切路径是正确的。然而,我还是得到了
Py4JJavaError: An error occurred while calling z:org.apache.spark.api.python.PythonRDD.collectAndServe.
: org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 1.0 failed 4 times, most recent failure: Lost task 0.3 in stage 1.0 (TID 14, spark-slave11.ydcloud.net): java.io.FileNotFoundException: File file:/<path to file> does not exist我运行的是分布式spark,而不是本地的。为什么会出现这个错误?
https://stackoverflow.com/questions/38210703
复制相似问题