目标失败

Question

• 什么Grafana版本和您正在使用的操作系统？

Promtail: Loki:2.2.0，Kubernetes (GitVersion:"v1.18.8")和Helm (版本：“v3.6.2”)

• What你想要实现吗？

可以刮掉我的目标然后把他们推到洛基。

• 发生了什么？

所有目标都被标记为“未准备好”。如果我要访问/targets页面，我的所有active_targets都被标记为"false“。在洛基我没有日志。此外，/var/log/文件夹在Promtail中也是空的。我从船尾舱收到的原木如下：

level=info ts=2021-08-06T05:30:09.076046169Z caller=filetargetmanager.go:254 msg="Adding target" key="{app=\"<app_name>\", container=\"<container_name>", job=\"<job_name>", namespace=\"<namesapce_name>", node_name=\"<nodeName_name>", pod=\"<pod_name>"}"

level=info ts=2021-08-06T05:30:09.076046169Z caller=filetargetmanager.go:254 msg="Removing target" key="{app=\"<app_name>\", container=\"<container_name>", job=\"<job_name>", namespace=\"<namesapce_name>", node_name=\"<nodeName_name>", pod=\"<pod_name>"}"

level=info ts=2021-08-06T05:30:14.095615824Z caller=filetarget.go:150 msg="filetarget: watcher closed, tailer stopped, positions saved" path=/var/log/pods/*<some_path>/<container_name>/*.log

Promtail/指标：

 HELP promtail_targets_failed_total Number of failed targets.
 TYPE promtail_targets_failed_total counter
promtail_targets_failed_total{reason="empty_labels"} 2280
promtail_targets_failed_total{reason="exists"} 470
 HELP request_duration_seconds Time (in seconds) spent serving HTTP requests.

• ，你期望会发生什么？

我的目标被刮了然后被推到洛基。

• 您能复制/粘贴您遇到问题的配置吗？

  file: |
    server:
      log_level: {{ .Values.config.logLevel }}
      http_listen_port: {{ .Values.config.serverPort }}
      health_check_target: false
    client:
      url: {{ tpl .Values.config.lokiAddress . }}
      {{- tpl .Values.config.snippets.extraClientConfigs . | nindent 2 }}
    positions:
      filename: /run/promtail/positions.yaml
    scrape_configs:
      {{- tpl .Values.config.snippets.scrapeConfigs . | nindent 2 }}
      {{- tpl .Values.config.snippets.extraScrapeConfigs . | nindent 2 }}

scrapeConfigs: |
      # See also https://github.com/grafana/loki/blob/master/production/ksonnet/promtail/scrape_config.libsonnet for reference
      # Pods with a label 'app.kubernetes.io/name'
      - job_name: kubernetes-pods-app-kubernetes-io-name
        pipeline_stages:
          {{- toYaml .Values.config.snippets.pipelineStages | nindent 4 }}
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_label_app_kubernetes_io_name
            target_label: app
          - action: drop
            regex: ''
            source_labels:
              - app
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_label_app_kubernetes_io_component
            target_label: component
          {{- if .Values.config.snippets.addScrapeJobLabel }}
          - action: replace
            replacement: kubernetes-pods-app-kubernetes-io-name
            target_label: scrape_job
          {{- end }}
          {{- toYaml .Values.config.snippets.common | nindent 4 }}
      # Pods with a label 'app'
      - job_name: kubernetes-pods-app
        pipeline_stages:
          {{- toYaml .Values.config.snippets.pipelineStages | nindent 4 }}
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          # Drop pods with label 'app.kubernetes.io/name'. They are already considered above
          - action: drop
            regex: .+
            source_labels:
              - __meta_kubernetes_pod_label_app_kubernetes_io_name
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_label_app
            target_label: app
          - action: drop
            regex: ''
            source_labels:
              - app
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_label_component
            target_label: component
          {{- if .Values.config.snippets.addScrapeJobLabel }}
          - action: replace
            replacement: kubernetes-pods-app
            target_label: scrape_job
          {{- end }}
          {{- toYaml .Values.config.snippets.common | nindent 4 }}
      # Pods with direct controllers, such as StatefulSet
      - job_name: kubernetes-pods-direct-controllers
        pipeline_stages:
          {{- toYaml .Values.config.snippets.pipelineStages | nindent 4 }}
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          # Drop pods with label 'app.kubernetes.io/name' or 'app'. They are already considered above
          - action: drop
            regex: .+
            separator: ''
            source_labels:
              - __meta_kubernetes_pod_label_app_kubernetes_io_name
              - __meta_kubernetes_pod_label_app
          - action: drop
            regex: '[0-9a-z-.]+-[0-9a-f]{8,10}'
            source_labels:
              - __meta_kubernetes_pod_controller_name
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_controller_name
            target_label: app
          {{- if .Values.config.snippets.addScrapeJobLabel }}
          - action: replace
            replacement: kubernetes-pods-direct-controllers
            target_label: scrape_job
          {{- end }}
          {{- toYaml .Values.config.snippets.common | nindent 4 }}
      # Pods with indirect controllers, such as Deployment
      - job_name: kubernetes-pods-indirect-controller
        pipeline_stages:
          {{- toYaml .Values.config.snippets.pipelineStages | nindent 4 }}
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          # Drop pods with label 'app.kubernetes.io/name' or 'app'. They are already considered above
          - action: drop
            regex: .+
            separator: ''
            source_labels:
              - __meta_kubernetes_pod_label_app_kubernetes_io_name
              - __meta_kubernetes_pod_label_app
          - action: keep
            regex: '[0-9a-z-.]+-[0-9a-f]{8,10}'
            source_labels:
              - __meta_kubernetes_pod_controller_name
          - action: replace
            regex: '([0-9a-z-.]+)-[0-9a-f]{8,10}'
            source_labels:
              - __meta_kubernetes_pod_controller_name
            target_label: app
          {{- if .Values.config.snippets.addScrapeJobLabel }}
          - action: replace
            replacement: kubernetes-pods-indirect-controller
            target_label: scrape_job
          {{- end }}
          {{- toYaml .Values.config.snippets.common | nindent 4 }}
      # All remaining pods not yet covered
      - job_name: kubernetes-other
        pipeline_stages:
          {{- toYaml .Values.config.snippets.pipelineStages | nindent 4 }}
        kubernetes_sd_configs:
          - role: pod
        relabel_configs:
          # Drop what has already been covered
          - action: drop
            regex: .+
            separator: ''
            source_labels:
              - __meta_kubernetes_pod_label_app_kubernetes_io_name
              - __meta_kubernetes_pod_label_app
          - action: drop
            regex: .+
            source_labels:
              - __meta_kubernetes_pod_controller_name
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_name
            target_label: app
          - action: replace
            source_labels:
              - __meta_kubernetes_pod_label_component
            target_label: component
          {{- if .Values.config.snippets.addScrapeJobLabel }}
          - action: replace
            replacement: kubernetes-other
            target_label: scrape_job
          {{- end }}
          {{- toYaml .Values.config.snippets.common | nindent 4 }}

• 您在Grafana UI或相关日志中收到任何错误吗？如果是的话，请告诉我们到底是什么。--

• ，你有没有遵守任何在线指示？如果是，URL是什么？

我主要是听从官方回购的指示。

https://github.com/grafana/helm-charts/tree/main/charts

我创建了以下资源：

为洛基:我有一个秘密(与信任)，服务和状态集。

我有一个DaemonSet，秘密，强大的ClusterRole和CluserRoleBinding。

Answer 1

我不确定这会对你有帮助，但我有相同的活动日志添加目标，并立即删除它们。

我的配置有点不同，我在本地运行promtail并抓取一些文件。问题是promtail没有读取这些文件的访问权限。

所以，我建议你再检查一下你的尾舱是否有读取文件的权限，然后重新启动服务。

我也没有在grafana或Loki中看到任何错误，因为日志从未被推到Loki。