ELK Stack -远程服务器的日志被正确解析和分类，但被转储到stout而不是elasticsearch

Question

正如(冗长的)主题所建议的，我的远程服务器将日志发送到logstash服务器。

下面是logstash的配置：

输入：

[root@tool01 conf.d]# cat 01-lumberjack-input.conf
input {
  lumberjack {
    port => 5000
    type => "logs"
    ssl_certificate => "/etc/pki/tls/certs/logstash-forwarder.crt"
    ssl_key => "/etc/pki/tls/private/logstash-forwarder.key"
  }
}

类型：

[root@tool01 conf.d]# cat 10-syslog.conf
    filter {
      if [type] == "syslog" {
        grok {
          match => { "message" => "%{SYSLOGTIMESTAMP:syslog_timestamp} %{SYSLOGHOST:syslog_hostname}     %{DATA:syslog_program}(?:\[%{POSINT:syslog_pid}\])?: %{GREEDYDATA:syslog_message}" }
          add_field => [ "received_at", "%{@timestamp}" ]
          add_field => [ "received_from", "%{host}" ]
        }
        syslog_pri { }
        date {
          match => [ "syslog_timestamp", "MMM  d HH:mm:ss", "MMM dd HH:mm:ss" ]
        }
      }
    }

输出：

[root@tool01 conf.d]# cat 30-lumberjack-output.conf
output {
  elasticsearch { host => localhost }
  stdout { codec => rubydebug }
}

本地日志工作正常，但远程的所有日志都被发送到stdout：

[root@tool01 logstash]# tail -n 19 logstash.stdout
{
             "message" => "Nov 29 20:16:44 foreman dhcpd: DHCPACK on 192.168.50.100 to 3c:4a:92:12:1c:cb via eth0",
            "@version" => "1",
          "@timestamp" => "2014-11-29T20:16:44.000Z",
                "type" => "syslog",
                "file" => "/var/log/messages",
                "host" => "foreman.ics.dmz",
              "offset" => "3511785",
    "syslog_timestamp" => "Nov 29 20:16:44",
     "syslog_hostname" => "foreman",
      "syslog_program" => "dhcpd",
      "syslog_message" => "DHCPACK on 192.168.50.100 to 3c:4a:92:12:1c:cb via eth0",
         "received_at" => "2014-11-29 07:16:46 UTC",
       "received_from" => "foreman.ics.dmz",
"syslog_severity_code" => 5,
"syslog_facility_code" => 1,
     "syslog_facility" => "user-level",
     "syslog_severity" => "notice"
}

这可能是一个愚蠢的问题，但是我只在客户端服务器上安装了shipper，我需要在那里也运行logstash吗？

提前感谢！

Answer 1

事情比我最初预想的更令人困惑。

例如，此日志：

Dec  2 10:39:12 foreman dhcpd: DHCPACK on 192.168.50.52 to 6c:ad:f8:26:b1:68 (Chromecast) via eth0

图表下的列表视图正确地列出了时间，但条形图上的条目是23:39:12？

13小时的延迟听起来令人难以置信，但13小时到秒听起来更像是一个时区问题？然而，所有运行的日期都与date验证的日期相同

Answer 2

更多信息：

[root@tool01 tmp]# curl -s XGET http://192.168.50.241:9200/logstash-2014.12.03/_search?pretty=true | tail -n 15; TZ=UTC date
{
  "_index": "logstash-2014.12.03",
  "_type": "syslog",
  "_id": "bOVXPpf9SFOKKPgx7PWfCA",
  "_score": 1.0,
  "_source": {
    "message": "Dec  3 00:06:30 foreman dhcpd: DHCPREQUEST for 192.168.50.251 from 00:15:5d:32:14:09 via eth0",
    "@version": "1",
    "@timestamp": "2014-12-03T00:06:30.000Z",
    "type": "syslog",
    "file": "/var/log/messages",
    "host": "foreman.ics.dmz",
    "offset": "3756888",
    "syslog_timestamp": "Dec  3 00:06:30",
    "syslog_hostname": "foreman",
    "syslog_program": "dhcpd",
    "syslog_message": "DHCPREQUEST for 192.168.50.251 from 00:15:5d:32:14:09 via eth0",
    "received_at": "2014-12-02 11:06:31 UTC",
    "received_from": "foreman.ics.dmz",
    "syslog_severity_code": 5,
    "syslog_facility_code": 1,
    "syslog_facility": "user-level",
    "syslog_severity": "notice"
  }
},
{
  "_index": "logstash-2014.12.03",
  "_type": "syslog",
  "_id": "5axCp7UgRxmunclFqlgKNw",
  "_score": 1.0,
  "_source": {
    "message": "Dec  3 00:06:30 foreman dhcpd: DHCPACK on 192.168.50.251 to 00:15:5d:32:14:09 via eth0",
    "@version": "1",
    "@timestamp": "2014-12-03T00:06:30.000Z",
    "type": "syslog",
    "file": "/var/log/messages",
    "host": "foreman.ics.dmz",
    "offset": "3756982",
    "syslog_timestamp": "Dec  3 00:06:30",
    "syslog_hostname": "foreman",
    "syslog_program": "dhcpd",
    "syslog_message": "DHCPACK on 192.168.50.251 to 00:15:5d:32:14:09 via eth0",
    "received_at": "2014-12-02 11:06:31 UTC",
    "received_from": "foreman.ics.dmz",
    "syslog_severity_code": 5,
    "syslog_facility_code": 1,
    "syslog_facility": "user-level",
    "syslog_severity": "notice"
  }
},
{
  "_index": "logstash-2014.12.03",
  "_type": "syslog",
  "_id": "m_MuAZPcS8ixmCn5getvyg",
  "_score": 1.0,
  "_source": {
    "message": "Dec  3 00:06:30 spacewalk dhclient[906]: DHCPREQUEST on eth0 to 192.168.50.240 port 67 (xid=0x421c37e1)",
    "@version": "1",
    "@timestamp": "2014-12-03T00:06:30.000Z",
    "type": "syslog",
    "file": "/var/log/messages",
    "host": "spacewalk.ics.dmz",
    "offset": "269907",
    "syslog_timestamp": "Dec  3 00:06:30",
    "syslog_hostname": "spacewalk",
    "syslog_program": "dhclient",
    "syslog_pid": "906",
    "syslog_message": "DHCPREQUEST on eth0 to 192.168.50.240 port 67 (xid=0x421c37e1)",
    "received_at": "2014-12-02 11:06:33 UTC",
    "received_from": "spacewalk.ics.dmz",
    "syslog_severity_code": 5,
    "syslog_facility_code": 1,
    "syslog_facility": "user-level",
    "syslog_severity": "notice"
  }
}
Tue Dec  2 19:19:14 UTC 2014

所以看起来数据在写入EL之前被缓冲了？