在Logstash中使用Gelf创建动态索引

Question

我使用logback与biz.paluch.logging.gelf.logback.GelfLogbackAppender一起登录。我目前有3个服务正在运行，我希望其中两个服务的输出以logstash-ingest的形式在logstash中运行，另一个以logstash-digest的形式运行。示例：

我希望这两个人都有索引logstash-ingest

服务1

<!DOCTYPE configuration>

<configuration>
    <contextName>test</contextName>
    <jmxConfigurator/>
    <appender name="gelf" class="biz.paluch.logging.gelf.logback.GelfLogbackAppender">
        <host>udp:localhost</host>
        <port>12201</port>
        <version>1.1</version>
        <extractStackTrace>true</extractStackTrace>
        <filterStackTrace>true</filterStackTrace>
        <timestampPattern>yyyy-MM-dd HH:mm:ss,SSS</timestampPattern>
        <maximumMessageSize>8192</maximumMessageSize>

        <param name="AdditionalFields" value="tag=example1-api,INDEX_PREFIX=ingest" />
        <param name="AdditionalFieldTypes" value="tag=String,INDEX_PREFIX=String" />

        <dynamicMdcFields>(field1|field2)</dynamicMdcFields>
    </appender>

    <logger name="com.example1" level="INFO" />

    <root level="INFO">
        <appender-ref ref="gelf" />
    </root>
</configuration>

服务2

<!DOCTYPE configuration>

<configuration>
    <contextName>test</contextName>
    <jmxConfigurator/>
    <appender name="gelf" class="biz.paluch.logging.gelf.logback.GelfLogbackAppender">
        <host>udp:localhost</host>
        <port>12201</port>
        <version>1.1</version>
        <extractStackTrace>true</extractStackTrace>
        <filterStackTrace>true</filterStackTrace>
        <timestampPattern>yyyy-MM-dd HH:mm:ss,SSS</timestampPattern>
        <maximumMessageSize>8192</maximumMessageSize>

        <param name="AdditionalFields" value="tag=example2-api,INDEX_PREFIX=ingest" />
        <param name="AdditionalFieldTypes" value="tag=String,INDEX_PREFIX=String" />

        <dynamicMdcFields>(field1|field2)</dynamicMdcFields>
    </appender>

    <logger name="com.example2" level="INFO" />

    <root level="INFO">
        <appender-ref ref="gelf" />
    </root>
</configuration>

第三个logstash-digest

服务3

<!DOCTYPE configuration>

<configuration>
    <contextName>test</contextName>
    <jmxConfigurator/>
    <appender name="gelf" class="biz.paluch.logging.gelf.logback.GelfLogbackAppender">
        <host>udp:localhost</host>
        <port>12201</port>
        <version>1.1</version>
        <extractStackTrace>true</extractStackTrace>
        <filterStackTrace>true</filterStackTrace>
        <timestampPattern>yyyy-MM-dd HH:mm:ss,SSS</timestampPattern>
        <maximumMessageSize>8192</maximumMessageSize>

        <param name="AdditionalFields" value="tag=example3-api,INDEX_PREFIX=digest" />
        <param name="AdditionalFieldTypes" value="tag=String,INDEX_PREFIX=String" />

        <dynamicMdcFields>(field1|field2)</dynamicMdcFields>
    </appender>

    <logger name="com.example3" level="INFO" />

    <root level="INFO">
        <appender-ref ref="gelf" />
    </root>
</configuration>

这是我的logstash.conf，但我不知道如何配置它，以便使用INDEX_PREFIX来区分这三个服务的输出。

input {

  gelf {
    id => "gelf"
    use_udp => true
    use_tcp => false
  }
}

## filters???

output {
    elasticsearch {
        hosts => ["es01:9200"]
        user => "elastic"
        password => "changeme"
        index => "logstash-{%}" ## what am I missing here?
    }
}

Answer 1

我一定是在吐露上花了很长时间，改变了太多的东西，最后我不知道什么是真正有效的。第二天，头脑清醒地再次尝试，显然我的问题的答案是index => "logstash-%{INDEX_PREFIX}" --我很确定我尝试过这个，但可能是结合了其他失败的东西。