PredictionIO引擎

Question

面对数据引擎迭代导致的错误，stackoverflow异常出现如下所示：

ERROR org.apache.spark.executor.Executor [Executor task launch worker-0] - Exception in task 0.0 in stage 30.0 (TID 76)
java.lang.StackOverflowError
    at java.io.ObjectInputStream$BlockDataInputStream.readByte(ObjectInputStream.java:2774)
    at java.io.ObjectInputStream.readHandle(ObjectInputStream.java:1450)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1512)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1774)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1924)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1801)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1351)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2000)

Answer 1

我得到了这个错误的解决方案：

1.只需在预测引擎中减少engine.json文件中算法的numIterations参数。

或者，如果这不起作用，请使用下面的另一个解决方案。

2.添加检查点，防止代码库使用的递归产生溢出。首先，创建一个新目录来存储检查点。然后，让您的SparkContext使用该目录来设置检查点。以下是Python中的示例：

sc.setCheckpointDir('checkpoint/')你可能还需要向ALS添加检查点，但我还不能确定这是否有区别。要在那里添加检查点(可能不是必需的)，只需执行以下操作：

ALS.checkpointInterval =2