单元运行时错误:无法反序列化减少输入键

Question

我正在尝试运行一个Insert Insert分区表，其中包含一个组。

查询

'set hive.exec.dynamic.partition=true;
 set hive.exec.dynamic.partition.mode=nonstrict;
 set hive.execution.engine=tez;
 INSERT OVERWRITE TABLE table1 PARTITION (date) select col1,CONCAT(COALESCE(substr(Cdate,1,4),'-'),'',COALESCE(substr(Cdate,6,2),'-'),'',COALESCE(substr(Cdate,9,2),'-')),col3,col4,'mobile-data',data,date 
from
(select col1,substr(CDate,1,10) as Cdate,u.col3 as col3,u.col4 as col4,date,sum(u.col5+u.col6) as data from json_table lateral view explode(json_table.mobiledata) oc as u 
where date >= from_unixtime(unix_timestamp()-4*60*60*24, 'yyyyMMdd')
group by col1,substr(CDate,1,10),u.col3,u.col4,date)t;'

错误：

'Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error: Unable to deserialize reduce input key from x1x50x48x49x53x48x56x49x57x0x1x117x115x101x114x49x0x1x129x51x122x35x1x65x78x84x32x72x65x76x32x83x101x114x118x105x99x101x0x1x99x111x109x46x100x115x105x46x97x110x116x46x115x101x114x118x101x114x0x1x109x111x98x105x108x101x45x100x97x116x97x0x255 with properties{columns=reducesinkkey0,reducesinkkey1,reducesinkkey2,reducesinkkey3,reducesinkkey4,reducesinkkey5, serialization.lib=org.apache.hadoop.hive.serde2.binarysortable.BinarySortableSerDe, serialization.sort.order=++++++, columns.types=string,string,string,string,string,string}'

我从未见过这样的错误。有人知道怎么处理这件事吗？

如果只使用查询的select部分(不包括insert insert已分区表)，则此查询运行良好。

Answer 1

我通过这个链接帮助解决了这个问题。

http://documentation.altiscale.com/hive-runtime-error-unable-to-deserialize-input-key

我所需要的只是一个设置上的改变。

'Set hive.optimize.sort.dynamic.partition = false;'

添加此设置后，查询(插入到分区表中)工作正常。