Logstash未写入Elasticsearch

Question

我遇到一个错误，当message属性包含分层消息数据时，Logstash没有将解析文档写入elasticsearch。当消息属性不包含分层数据时，它可以很好地工作。以下是一些有效的数据：

{
  "Layer": "Web",
  "DurationMilliseconds": 65,
  "CreatedOn": "2014-09-29T20:44:40.5380157Z",
  "Enviroment": "Dev",
  "AssemblyName": "LoggingTest",
  "ClassName": "HomeController",
  "MethodName": "Index",
  "WindowsIdentity": "XXX\\YYY",
  "SessionId": "wrm11rken3lc442humrxyhoe",
  "Application": "LoggingTest",
  "Machine": "XXX.XXX.XXX.XXX",
  "Browser": "Chrome",
  "@version": "1",
  "@timestamp": "2014-09-29T20:45:38.432Z",
  "type": "Perf"
}

以下是一些不起作用的数据：

{
  "Enviroment": "Dev",
  "Level": "Fatal",
  "CreatedOn": "2014-09-29 20:46:30.5042",
  "WindowsIdentity": "XXX\\XXX",
  "Application": "LoggingTest",
  "UserAgent": "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36",
  "SessionId": "wrm11rken3lc442humrxyhoe",
  "URL": "/LoggingTest/jsnlog.logger",
  "UserAddress": "XXX.XXX.XXX.XXX",
  "Message": {
    "stack": "TypeError: undefined is not a function\n    at Log (http://XXX/LoggingTest/:58:16)\n    at HTMLInputElement.onclick (http://XXX/LoggingTest/:66:141)",
    "message": "undefined is not a function",
    "name": "TypeError",
    "logData": "JS Fatal Exception"
  },
  "@version": "1",
  "@timestamp": "2014-09-29T20:46:30.331Z",
  "type": "JS"
}

下面是我的logstash配置：

input {
    redis {
        host => "127.0.0.1"
        type => "JS"
        data_type => "list"
        key => "JS"
    }
}

filter
{
    json{ source => "message"}
}

output {
    stdout { codec => rubydebug}

    elasticsearch { 
        host => localhost 
        index => dev
    }
}

当我运行上面的代码时，第一个项目被成功解析到我的elasticsearch中，但第二个项目消失了，并且logstash没有写入任何错误。

Answer 1

我把你的日志改了。这是我的配置，

input {
    stdin{}
}

filter{
    json{ source => "message"}
}

output {
    stdout {
            codec => "rubydebug"
    }
    elasticsearch {
            host => localhost
            cluster => "BENLIM"
    }
}

使用此配置，当我将不起作用的日志发送到logstash时，它可以解析并输出到elasticsearch。仅供参考。