使用Logstash 7.2.0的Snow API

Question

我正在尝试使用Logstash 7.2.0中的以下配置查询snow API，

   input {
  http_poller {
    urls => {
    snowinc => {
    url => "https://l.service-now.com/api/now/table/incident?sysparm_limit=10"
    user => "logstash_user"
    password => "hello123"
    headers => {Accept => "application/json"}
    }
}
request_timeout => 60
metadata_target => "http_poller_metadata"
schedule => { cron => "* * * * * UTC"}
codec => "json"
}
}
output {
  elasticsearch {
    hosts => ["10.116.55.24:9200"]
    index => "snowinc"
}
        stdout { codec => rubydebug }
}

但是，我在索引中看不到任何数据。(索引是在没有数据的情况下创建的)当我尝试卷曲url时，我可以看到url中的数据。因此，我确信在API URL中有数据。

日志显示以下错误：

[2019-12-24T01:28:01,510][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"snowinc", :_type=>"_doc", :routing=>nil}, #LogStash::Event:0x13fbd67c], :response=>{"index"=>{"_index"=>"snowinc", "_type"=>"_doc", "id"=>"OkiXNm8Be2NFZrotRzc", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"mapper [result.assigned_to] of different type, current_type [text], merged_type [ObjectMapper]"}}}}

请帮忙找出我哪里错了。

提前感谢

Answer 1

问题出在URL数据本身，它有一个不允许正确转换的列的reference_link。

用一个sysparm_exclude_reference_link=true修改了网址，它就像奇迹一样工作！

谢谢你的帮助。

Answer 2

尝试使用过滤器http插件，如下所示

filter {
http {
    url => "https://service-now.com/api/now/table/incident"
    verb => "GET"
    user => "logstash_user"
    password => "user222"
    query => {
          "sysparm_limit" => 10
        }
        target_body => "[snowData]"
    headers => {
    "Content-Type" => "application/json"
    Accept => "application/json"
    }
    }
}