使用PIG进行电影数据集分析

Question

我有一个电影数据库的以下数据集：

评级: UserID，MovieID，评级:：Movies: MovieID，标题:：用户: UserID，性别，年龄

现在我必须加入上面的3个数据集，并确定哪部电影在女性中评分最高，在男性中评分最低，反之亦然。我已经完成了连接：

myusers = LOAD '/user/cloudera/movies/input/users.dat' 
  USING PigStorage(':') 
  AS (user:int, n1, gender:chararray, n2, age:int);

ratings = LOAD '/user/cloudera/movies/input/ratings.dat' 
  USING PigStorage(':') 
  AS (user:int, n1, movie:int, n2, rating:int);

movies = LOAD '/user/cloudera/movies/input/movies.dat' 
  USING PigStorage(':') 
  AS (movie:int,n1,title:chararray);

data = JOIN ratings BY user, myusers BY user;
data2= JOIN data BY ratings::movie, movies BY movie;

但在此之后，当我尝试从data2打印列时，我遇到了许多问题，例如“错误0:标量在输出中有多行”。有什么办法能帮我完成这项任务吗？

Answer 1

在下面的步骤之后

data = JOIN ratings BY user, myusers BY user;

通过使用性别作为数据集的filter.Order，创建两个数据集，一个用于男性，另一个用于女性，并获取这两个数据集的最大值和最小值。

male = FILTER data by gender == 'M'; -- Use the gender value for male
female = FILTER data by gender == 'F';
m_max = LIMIT (ORDER male by rating DESC) 1;
f_max = LIMIT (ORDER female by rating DESC) 1;
m_min = LIMIT (ORDER male by rating ASC) 1;
f_min = LIMIT (ORDER female by rating ASC) 1;