无法将数据从Apache hive加载到ElasticSearch -

Question

我使用的是CDH5.5，ElasticSearch-2.4.1。我已经创建了配置单元表，并尝试使用以下查询将配置单元表数据推送到ElasticSearch。

CREATE EXTERNAL TABLE test1_es(
  id string,
  timestamp string, 
  dept string)<br>
ROW FORMAT SERDE 'org.elasticsearch.hadoop.hive.EsSerDe'  
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'  
LOCATION
  'hdfs://quickstart.cloudera:8020/user/cloudera/elasticsearch/test1_es'
TBLPROPERTIES (  'es.nodes'='localhost', 
'es.resource'='sample/test1',
'es.mapping.names' = 'timestamp:@timestamp',
'es.port' = '9200', 
'es.input.json' = 'false', 
'es.write.operation' = 'index', 
'es.index.auto.create' = 'yes'
);<br>
INSERT INTO TABLE default.test1_es select id,timestamp,dept from test1_hive;

我在工作跟踪器URL中收到以下错误“

 Failed while trying to construct the redirect url to the log server. Log Server url may not be configured. <br>
java.lang.Exception: Unknown container. Container either has not started or has already completed or doesn't belong to this node at all. "

它将在hive终端中抛出"FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask"。

我尝试了论坛中提到的所有步骤，比如将/usr/lib/hive/bin/elasticsearch-hadoop-2.0.2.jar包含在hive-site.xml中，将ES-hadoop jar添加到/usr/lib/hadoop/elasticsearch-yarn-2.1.0.Beta3.jar中，将yarn jar复制到HIVEAUXJARS_PATH中。请建议我如何修复这个错误。

先谢谢你，Sreenath

Answer 1

我正在处理同样的问题，我发现hive抛出的执行错误是由无法解析的字符串类型的时间戳字段引起的。我想知道字符串类型的时间戳字段是否可以正确地映射到es，如果不能，这可能是根本原因。

顺便说一句，您应该转到hadoop MR日志以查找有关该错误的更多详细信息。

Answer 2

REATE EXTERNAL TABLE test1_es(
  id string,
  timestamp string, 
  dept string)<br>
ROW FORMAT SERDE 'org.elasticsearch.hadoop.hive.EsSerDe'  
STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'  
TBLPROPERTIES ...........

不需要位置