无法从给定路径读取架构:hdfs://.avsc

Question

我试图通过以下步骤创建一个蜂巢表：

1. 使用sqoop将数据加载到hdfs (已完成)
2. sqoop还创建了一个avsc文件，我将其上传到hdfs。
3. 在单元格中，我希望使用以下语句创建一个表：

命令：

CREATE EXTERNAL TABLE kontoauszug
ROW FORMAT SERDE 'org.apache.hadoop.hive.serde2.avro.AvroSerDe' 
STORED AS 
INPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerInputFormat'
OUTPUTFORMAT 'org.apache.hadoop.hive.ql.io.avro.AvroContainerOutputFormat'
LOCATION 'hdfs:///user/tki/KONTOAUSZUG'
TBLPROPERTIES ('avro.schema.url'='hdfs://m1.hdp2/user/tki/KONTOAUSZUG.avsc');

我收到以下错误：

FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. 
java.lang.RuntimeException: MetaException(message:org.apache.hadoop.hive.serde2.SerDeException 
Encountered AvroSerdeException determining schema. 
Returning signal schema to indicate problem: Unable to read schema from given path: hdfs://m1.hdp2/user/tki/KONTOAUSZUG.avsc)

这是否意味着，没有找到KONTOAUSZUG.avsc？我把它弄到手了。

其内容如下：

[hadoop@m1 hive]$ cat KONTOAUSZUG.avsc 
{
  "type" : "record",
  "name" : "KONTOAUSZUG",
  "doc" : "Sqoop import of KONTOAUSZUG",
  "fields" : [ {
    "name" : "FK_PROCESS_ID_INS",
    "type" : [ "null", "string" ],
    "default" : null,
    "columnName" : "FK_PROCESS_ID_INS",
    "sqlType" : "2"
  }, {
    "name" : "FK_SOURCE_ID",
    "type" : [ "null", "string" ],
    "default" : null,
    "columnName" : "FK_SOURCE_ID",
    "sqlType" : "2"
  }, {
    "name" : "SRC_STM_ID",
    "type" : [ "null", "string" ],
    "default" : null,
    "columnName" : "SRC_STM_ID",
    "sqlType" : "2"
  }, {
    "name" : "FK_PROCESS_ID_UPD",
    "type" : [ "null", "string" ],
    "default" : null,
    "columnName" : "FK_PROCESS_ID_UPD",
    "sqlType" : "2"
  }, {
    "name" : "BUCHUNGSDATUM",
    "type" : [ "null", "long" ],
    "default" : null,
    "columnName" : "BUCHUNGSDATUM",
    "sqlType" : "93"
  }, {
    "name" : "BUCHUNGSTEXT",
    "type" : [ "null", "string" ],
    "default" : null,
    "columnName" : "BUCHUNGSTEXT",
    "sqlType" : "12"
  }, {
    "name" : "SOLL",
    "type" : [ "null", "string" ],
    "default" : null,
    "columnName" : "SOLL",
    "sqlType" : "2"
  }, {
    "name" : "HABEN",
    "type" : [ "null", "string" ],
    "default" : null,
    "columnName" : "HABEN",
    "sqlType" : "2"
  }, {
    "name" : "FK_KONTO_ID",
    "type" : [ "null", "string" ],
    "default" : null,
    "columnName" : "FK_KONTO_ID",
    "sqlType" : "2"
  }, {
    "name" : "EINGABE_MANUELL_F",
    "type" : [ "null", "string" ],
    "default" : null,
    "columnName" : "EINGABE_MANUELL_F",
    "sqlType" : "2"
  } ],
  "tableName" : "KONTOAUSZUG"
}

Answer 1

在CREATE表语句中，您使用三个斜杠：hdfs:/，而不是hdfs://

(.)“hdfs://user/tki/KONTOAUSZUG‘TBLPROPERTIES (.)

此外，如果从namenode服务器获取模式，我认为您应该在主机url之后编写端口。

'avro.schema.url'='hdfs://m1.hdp2:port/user/tki/KONTOAUSZUG.avsc‘

有关详细信息，请查看这。还有，这是关于hdfs端口的答案。

Answer 2

这是一个访问错误。请检查avsc目录以获得适当的权限，并重试hdfs://m1.hdp2/user/tki/KONTOAUSZUG.avsc