闪烁序列化错误

Question

我正在尝试在我的Apache Flink Gelly图上运行标签传播协议。

下面是我的代码：

        Graph<String, Long, String> ugraph = Graph.fromDataSet(vertex, edgeSet, env).getUndirected();
        DataSet<Tuple2<String, Long>> idsWithInitialLabels = DataSetUtils
            .zipWithUniqueId(graph.getVertexIds())
            .map(new MapFunction<Tuple2<Long, String>, Tuple2<String, Long>>() {
                public Tuple2<String, Long> map(Tuple2<Long, String> tuple2) throws Exception {
                    return new Tuple2<String, Long>(tuple2.f1, tuple2.f0);
                }
            }); 
        DataSet<Vertex<String, Long>> verticesWithCommunity = graph.joinWithVertices(idsWithInitialLabels,
            new VertexJoinFunction<Long, Long>() {
            public Long vertexJoin(Long vertexValue, Long inputValue) {
            return inputValue;
        }})
    .run(new LabelPropagation<String, Long, String>(10)); 

我收到以下错误消息：

org.apache.flink.api.common.InvalidProgramException:对象org.apache.flink.graph.Graph$ApplyCoGroupToVertexValues@4dde0543在org.apache.flink.api.java.ClosureCleaner.ensureSerializable(ClosureCleaner.java:99)、org.apache.flink.api.java.ClosureCleaner.clean(ClosureCleaner.java:61)、org.apache.flink.api.java.DataSet.clean(DataSet.java:186)、org.apache.flink.api.java.operators.CoGroupOperator$CoGroupOperatorSets$CoGroupOperatorSetsPredicate$CoGroupOperatorWithoutFunction.with(CoGroupOperator.java:619)无法序列化at org.apache.flink.graph.Graph.joinWithVertices(Graph.java:587) at tu.master.ConceptDetection.TextProcessor.clustering(TextProcessor.java:405) at tu.master.ConceptDetection.TextProcessor$4.actionPerformed(TextProcessor.java:210)

感谢您的帮助:)

Answer 1

我猜包含图形代码的类不是Serializable。Java语言中的匿名类实际上是非静态的内部类，这意味着它们引用了包含类的this (参见this answer)。如果包含类不是Serializable，则this引用不会序列化，匿名类也不会序列化。

这就解释了为什么切换到lambda表达式会使其序列化。Lambda表达式不是匿名类，因此它们不会自动捕获隐式this引用。

它没有解释的是，为什么将MapFunction声明为匿名类仍然有效。如果你还有这段代码，@Nesrine，我很好奇整个类是什么样子的。