不能正确地使用promtail解析登录

Question

有人能就如何正确配置promtail来解析我的日志中的json输出提供建议吗？我还没有弄清楚promtail为什么没有识别json属性并在Grafana中显示它们。

我的logback附录，在logback.xml中：

<appender name="json"
    class="ch.qos.logback.core.ConsoleAppender">
    <layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
        <jsonFormatter
            class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
            <prettyPrint>true</prettyPrint>
        </jsonFormatter>
        <timestampFormat>yyyy-MM-dd' 'HH:mm:ss.SSS</timestampFormat>
    </layout>
</appender>

promtail.yaml

        ...
        ...
        ...
 scrape_configs:
    - job_name: parse-logs
      pipeline_stages:
        - docker: {}
        - json:
            expressions:
              subscriptionType:
              AuthenticationSuccess:
              user:
              execution_lasted: execution_lasted
              time-to-pull(.ms): time-to-pull(.ms)
            source: message
        - json:
            expressions:
              logger: logger
              level: level
              timestamp: timestamp
              message: message
        - labels:
            level:
            timestamp:
        ...
        ...
        ...

日志：

...
{
  "timestamp" : "2022-11-24 09:33:03.361",
  "level" : "DEBUG",
  "thread" : "http-nio-8081-exec-3",
  "logger" : "com.vanitysoft.boundariesio.unitedstates.domain.dao.impl.PostalGrabberDAOImpl",
  "message" : "{\"time-to-pull(.ms)\":12,\"user\":\"unknown\",\"number-of-postals\":1}",
  "context" : "default"
}{
  "timestamp" : "2022-11-24 09:33:03.361",
  "level" : "DEBUG",
  "thread" : "http-nio-8081-exec-3",
  "logger" : "com.vanitysoft.boundariesio.unitedstates.domain.dao.impl.PostalGrabberDAOImpl",
  "message" : "{\"time-to-pull(.ms)\":12,\"user\":\"unknown\",\"number-of-postals\":1}",
  "context" : "default"
}

在编辑promtail配置之后，我准备了一个汇总更新..。

kubectl edit configmap loki-promtail --namespace loki-stack
kubectl rollout restart deployment/lke-monitor-grafana

更新 promtail解析要求将日志写入同一行(没有漂亮的打印！)

<layout class="ch.qos.logback.contrib.json.classic.JsonLayout">
    <jsonFormatter          class="ch.qos.logback.contrib.jackson.JacksonJsonFormatter">
    </jsonFormatter>
     <appendLineSeparator>true</appendLineSeparator>
    <timestampFormat>yyyy-MM-dd' 'HH:mm:ss.SSS</timestampFormat>
</layout>

Answer 1

假设您正确地复制到您的promtail.yaml中，则存在格式化错误：-docker: {}应该是- docker: {}

---
scrape_configs:
  - job_name: parse-logs
    pipeline_stages:
      - docker: {}
      - json:
          expressions:
            subscriptionType: null
            AuthenticationSuccess: null
            user: null
            execution_lasted: execution_lasted
            time-to-pull(.ms): time-to-pull(.ms)
          source: message
      - json:
          expressions:
            logger: logger
            level: level
            timestamp: timestamp
            message: message
      - labels:
          level: null
          timestamp: null