FlatMapFunction没有定义serialversionUid

Question

​

@Component("loader")
    @Slf4j
    public class LoaderSpark implements Serializable , SparkJob {
            private static final int MAX_VERSIONS = 1;
            private static final long serialVersionUID = 5911491576792653184L;
           spark.sparkContext().getConf().toDebugString());
          HBaseUtil hBaseUtil = new HBaseUtil(props);

        byte[][] prefixes = new byte[][]{toBytes("document"),
                toBytes("dataSource"),
                toBytes("hold:")};

        Filter filter = new MultipleColumnPrefixFilter(prefixes);

        Scan scan = new Scan();
        scan.addFamily(toBytes("data"));        
        scan.setFilter(filter);

    @Override
    public void run(final SparkSession spark, final String... args) {

        JavaPairRDD<ImmutableBytesWritable, Result> ScanRDD
                = hBaseUtil.createScanRdd(spark, "Test", scan);

        ScanRDD.mapToPair(new FlatMapFunction<Tuple2<ImmutableBytesWritable,Result>, String, String>() {


            @Override
            public Iterator<String> call(Tuple2<ImmutableBytesWritable, Result> immutableBytesWritableResultTuple2) throws Exception {
                return null;
            }

        });

我的ScanRDD.mapToPair函数报告缺少serialversionUid，并要求在ScanRDD.mapToPair(new FlatMapFunction<Tuple2<ImmutableBytesWritable,Result>, String, String>() {中添加一个

我想知道为什么我需要在我的FlatMapFunction中添加serialversionUid？我已经为这门课准备了一个。

有什么帮助吗？

Answer 1

这背后的原因只是一个静态分析，它验证了实现Serializable的每个类实例都有serialVersionUID字段。

在本例中，FlatMapFunction实现了java.io.Serializable，而您正在为它创建一个匿名类。

所以只需添加serialVersionUID

ScanRDD.mapToPair(new FlatMapFunction<Tuple2<ImmutableBytesWritable,Result>, String, String>() {
//inside the anonymous class
private final serialVersionUID = 1L;

@Override
public Iterator<String> call(Tuple2<ImmutableBytesWritable, Result> immutableBytesWritableResultTuple2) throws Exception {
                return null;
}

})

请参阅https://spark.apache.org/docs/2.2.2/api/java/org/apache/spark/api/java/function/FlatMapFunction.html