EFK (Elasticsearch+Fluentd-(td-agent)+ Kibana )：Kibana没有显示正确的日志

Question

我已经在redhat linux系统(7.6版)上安装了EFK。fluentd的稳定分布，即使用td-agent代替fluentd。在td-agent.conf文件中配置日志文件的路径(/mnt/ log /Startup.log)。但在kibana仪表板上，它会显示td-agent.log中的内容，而不是日志文件(Startup.log)。

td-agent.log：

2020-09-04 16:02:16 +0530 [info]: parsing config file is succeeded path="/etc/td-agent/td-agent.conf"
2020-09-04 16:02:16 +0530 [info]: gem 'fluent-plugin-elasticsearch' version '4.0.9'
2020-09-04 16:02:16 +0530 [info]: gem 'fluent-plugin-kafka' version '0.13.0'
2020-09-04 16:02:16 +0530 [info]: gem 'fluent-plugin-prometheus' version '1.8.0'
2020-09-04 16:02:16 +0530 [info]: gem 'fluent-plugin-prometheus_pushgateway' version '0.0.2'
2020-09-04 16:02:16 +0530 [info]: gem 'fluent-plugin-record-modifier' version '2.1.0'
2020-09-04 16:02:16 +0530 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '2.3.0'
2020-09-04 16:02:16 +0530 [info]: gem 'fluent-plugin-s3' version '1.3.2'
2020-09-04 16:02:16 +0530 [info]: gem 'fluent-plugin-systemd' version '1.0.2'
2020-09-04 16:02:16 +0530 [info]: gem 'fluent-plugin-td' version '1.1.0'
2020-09-04 16:02:16 +0530 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.4'
2020-09-04 16:02:16 +0530 [info]: gem 'fluent-plugin-webhdfs' version '1.2.5'
2020-09-04 16:02:16 +0530 [info]: gem 'fluentd' version '1.11.1'
2020-09-04 16:02:16 +0530 [info]: 'flush_interval' is configured at out side of <buffer>. 'flush_mode' is set to 'interval' to keep existing behaviour
2020-09-04 16:02:16 +0530 [debug]: 'host localhost' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'host: localhost' doesn't have tag placeholder
2020-09-04 16:02:16 +0530 [debug]: 'index_name fluentd' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'index_name: fluentd' doesn't have tag placeholder
2020-09-04 16:02:16 +0530 [debug]: 'template_name ' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'template_name: ' doesn't have tag placeholder
2020-09-04 16:02:16 +0530 [debug]: 'logstash_prefix logstash' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_prefix: logstash' doesn't have tag placeholder
2020-09-04 16:02:16 +0530 [debug]: 'logstash_dateformat %Y.%m.%d' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_dateformat: %Y.%m.%d' has timestamp placeholders, but chunk key 'time' is not configured
2020-09-04 16:02:16 +0530 [debug]: 'logstash_dateformat %Y.%m.%d' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_dateformat: %Y.%m.%d' doesn't have tag placeholder
2020-09-04 16:02:16 +0530 [debug]: 'deflector_alias ' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'deflector_alias: ' doesn't have tag placeholder
2020-09-04 16:02:16 +0530 [debug]: 'application_name default' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'application_name: default' doesn't have tag placeholder
2020-09-04 16:02:16 +0530 [debug]: 'ilm_policy_id logstash-policy' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'ilm_policy_id: logstash-policy' doesn't have tag placeholder
2020-09-04 16:02:16 +0530 [debug]: Need substitution: false
2020-09-04 16:02:16 +0530 [debug]: 'host_placeholder localhost' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'host_placeholder: localhost' doesn't have tag placeholder
2020-09-04 16:02:16 +0530 [warn]: define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead
2020-09-04 16:02:16 +0530 [info]: using configuration file: <ROOT>
  <system>
    log_level debug
  </system>
  <source>
    @type tail
    path "/mnt/Log/Startup.log"
    pos_file "/mnt/Log/Startup.log.pos"
    format multiline
    format_firstline /\d{4}-\d{1,2}-\d{1,2}/
    format1 /^(?<time>\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2}) \[(?<thread>.*)\] (?<level>[^\s]+)(?<message>.*)/
    tag "log"
    <parse>
      format_firstline /\d{4}-\d{1,2}-\d{1,2}/
      @type multiline
      format1 /^(?<time>\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2}) \[(?<thread>.*)\] (?<level>[^\s]+)(?<message>.*)/
      unmatched_lines
    </parse>
  </source>
  <match *.**>
    @type elasticsearch
    host "localhost"
    port 9200
    include_tag_key true
    tag_key "@log_name"
    logstash_format true
    flush_interval 10s
    <buffer>
      flush_interval 10s
    </buffer>
  </match>
</ROOT>
2020-09-04 16:02:16 +0530 [info]: starting fluentd-1.11.1 pid=67918 ruby="2.4.10"
2020-09-04 16:02:16 +0530 [info]: spawn command to main:  cmdline=["/opt/td-agent/embedded/bin/ruby", "-Eascii-8bit:ascii-8bit", "/opt/td-agent/embedded/bin/fluentd", "--log", "/var/log/td-agent/td-agent.log", "--daemon", "/var/run/td-agent/td-agent.pid", "--under-supervisor"]
2020-09-04 16:02:17 +0530 [info]: adding match pattern="*.**" type="elasticsearch"
2020-09-04 16:02:17 +0530 [info]: #0 'flush_interval' is configured at out side of <buffer>. 'flush_mode' is set to 'interval' to keep existing behaviour
2020-09-04 16:02:17 +0530 [debug]: #0 'host localhost' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'host: localhost' doesn't have tag placeholder
2020-09-04 16:02:17 +0530 [debug]: #0 'index_name fluentd' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'index_name: fluentd' doesn't have tag placeholder
2020-09-04 16:02:17 +0530 [debug]: #0 'template_name ' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'template_name: ' doesn't have tag placeholder
2020-09-04 16:02:17 +0530 [debug]: #0 'logstash_prefix logstash' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_prefix: logstash' doesn't have tag placeholder
2020-09-04 16:02:17 +0530 [debug]: #0 'logstash_dateformat %Y.%m.%d' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_dateformat: %Y.%m.%d' has timestamp placeholders, but chunk key 'time' is not configured
2020-09-04 16:02:17 +0530 [debug]: #0 'logstash_dateformat %Y.%m.%d' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'logstash_dateformat: %Y.%m.%d' doesn't have tag placeholder
2020-09-04 16:02:17 +0530 [debug]: #0 'deflector_alias ' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'deflector_alias: ' doesn't have tag placeholder
2020-09-04 16:02:17 +0530 [debug]: #0 'application_name default' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'application_name: default' doesn't have tag placeholder
2020-09-04 16:02:17 +0530 [debug]: #0 'ilm_policy_id logstash-policy' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'ilm_policy_id: logstash-policy' doesn't have tag placeholder
2020-09-04 16:02:17 +0530 [debug]: #0 Need substitution: false
2020-09-04 16:02:17 +0530 [debug]: #0 'host_placeholder localhost' is tested built-in placeholder(s) but there is no valid placeholder(s). error: Parameter 'host_placeholder: localhost' doesn't have tag placeholder
2020-09-04 16:02:17 +0530 [warn]: #0 Detected ES 7.x: `_doc` will be used as the document `_type`.
2020-09-04 16:02:17 +0530 [info]: adding source type="tail"
2020-09-04 16:02:17 +0530 [warn]: #0 define <match fluent.**> to capture fluentd logs in top level is deprecated. Use <label @FLUENT_LOG> instead
2020-09-04 16:02:17 +0530 [info]: #0 starting fluentd worker pid=67935 ppid=67930 worker=0
2020-09-04 16:02:17 +0530 [debug]: #0 buffer started instance=70139276565080 stage_size=0 queue_size=0
2020-09-04 16:02:17 +0530 [debug]: #0 enqueue_thread actually running
2020-09-04 16:02:17 +0530 [debug]: #0 tailing paths: target = /mnt/Log/Startup.log | existing =
2020-09-04 16:02:17 +0530 [info]: #0 following tail of /mnt/Log/Startup.log
2020-09-04 16:02:17 +0530 [info]: #0 fluentd worker is now running worker=0
2020-09-04 16:02:17 +0530 [debug]: #0 flush_thread actually running

尽管上面的日志显示它正在拖尾Startup.log。它仍然在kibana仪表板上显示td-agent.log中的内容，而不是配置的日志文件中的内容。此外，我能够找到kibana上的默认索引

td-agent.conf：

<system>
log_level debug
</system>

<source>
@type tail
path /mnt/Log/Startup.log
pos_file /mnt/Log/Startup.log.pos
format multiline
format_firstline /\d{4}-\d{1,2}-\d{1,2}/
format1 /^(?<time>\d{4}-\d{1,2}-\d{1,2} \d{1,2}:\d{1,2}:\d{1,2}) \[(?<thread>.*)\] (?<level>[^\s]+)(?<message>.*)/
tag log
</source>

<match *.**>
   @type elasticsearch
    host localhost
    port 9200
    include_tag_key true
    tag_key @log_name
    logstash_format true
    flush_interval 10s
</match>

版本详细信息：

Elasticsearch版本7.8.0 Fluentd (td代理)版本3.8.0 Kibana版本7.8.0

我已经使用rpms安装了上述所有工具：

elasticsearch-7.8.1-x86_64.rpm td-agent-3.8.0-0.el7.x86_64.rpm kibana-7.8.1-x86_64.rpm

curl -X GET "localhost:9200/?pretty“提供：

{
  "name" : "ncnsidapp2",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "MnfFXTyZT0ahp9u4eLSW2A",
  "version" : {
    "number" : "7.8.1",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "b5ca9c58fb664ca8bf9e4057fc229b3396bf3a89",
    "build_date" : "2020-07-21T16:40:44.668009Z",
    "build_snapshot" : false,
    "lucene_version" : "8.5.1",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

我能够在kibana上找到默认的索引。

这里的问题可能是什么？请帮帮忙。

Answer 1

尝试在您的部分中设置logstash_prefix。当您将logstash_format设置为true时，它将覆盖您设置为索引名称的内容。如果设置为logstash_prefix，则默认情况下，索引名称的格式为{logstash_prefix}-%y.%m.%d。