Apache Pig的解释函数

Question

我正在试着理解猪的explain-function (link)。

A = load 'numbers' using PigStorage(',') as (name, age);
explain A;

这给了我：

#-----------------------------------------------
# New Logical Plan:
#-----------------------------------------------
A: (Name: LOStore Schema: name#5:bytearray,age#6:bytearray)
|
|---A: (Name: LOLoad Schema: name#5:bytearray,age#6:bytearray)RequiredFields:[0, 1]
#-----------------------------------------------
# Physical Plan:
#-----------------------------------------------
A: Store(fakefile:org.apache.pig.builtin.PigStorage) - scope-1
|
|---A: Load(file:///...pig-0.14.0/numbers:PigStorage(',')) - scope-0

2014-12-07 15:07:10,596 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2014-12-07 15:07:10,609 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2014-12-07 15:07:10,610 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
#--------------------------------------------------
# Map Reduce Plan                                  
#--------------------------------------------------
MapReduce node scope-2
Map Plan
A: Store(fakefile:org.apache.pig.builtin.PigStorage) - scope-1
|
|---A: Load(file:///.../pig-0.14.0/numbers:PigStorage(',')) - scope-0--------
Global sort: false
----------------

Answer 1

它会告诉你小猪的步数。在你的例子中，它解释了别名A是如何通过加载数据来填充的，因为你还没有对它做任何事情，所以它进入了"fakefile“。它既描述了流，也描述了它是如何进入map/reduce的

正如你可能亲眼所见，它变得非常混乱，非常快。你可能想看看Netflix的Lipstick，看看更容易上手的东西