expr: sum(kafka_topic_partition_in_sync_replica) by (topic) < 1
参数 即将部署的kafka-exporter要从kafka取得数据,因此要准备kafka-exporter参数; 查看kafka的服务中TYPE是ClusterIP的那个,如下图红框所示: ? 上述红框中的服务名字是kafka、端口是9092,因此稍后在kafka-exporter中配置的kafka信息就是kafka:9092 实际操作 添加Helm仓库(该仓库中有我们需要的kafka-exporter ):helm repo add gkarthiks https://gkarthiks.github.io/helm-charts 下载kafka-exporter:helm fetch gkarthiks 在values.yaml文件所在目录执行:helm install --name-template kafka-exporter -f values.yaml . 此时我们部署好了kafka-exporter,能给prometheus提供监控数据了,但prometheus并不知道要来这里采集数据,因此接下来要配置prometheus; 确认kafka-exporter
metrics: kafka: enabled: true image: registry: myharbor.com repository: bigdata/kafka-exporter docker push myharbor.com/bigdata/kafka:3.2.1-debian-11-r16 # node-export docker pull docker.io/bitnami/kafka-exporter :1.6.0-debian-11-r8 docker tag docker.io/bitnami/kafka-exporter:1.6.0-debian-11-r8 myharbor.com/bigdata /kafka-exporter:1.6.0-debian-11-r8 docker push myharbor.com/bigdata/kafka-exporter:1.6.0-debian-11-r8
prometheus配置监控 Kafka 监控组件:kafka-exporter github地址:GitHub - imduffy15/kafka_exporter: Kafka exporter for =on-failure:5 \ --name kafka_exporter \ -v /etc/localtime:/etc/localtime \ -p 9308:9308 \ danielqsj/kafka-exporter description: "{{$.Labels.consumergroup}}##{{$.Labels.topic}}:消费滞后超过1000持续3分钟(当前:{{$value}})" - alert: kafka-exporter labels: serverity: warning status: 严重 annotations: summary: "kafka-exporter down" description: "kafka-exporter down {{$.Labels.instance}}" - alert: kafka server
生态丰富 无缝对接prometheus、grafana grafana有大量开源的DashBoard配置 4、kafka官方项目 KIP-575: build a Kafka-Exporter by Java kafka后面可能会推出官方的Kafka-Exporter 二、实现分析 1、方案架构 kafka exporter代码层借助大量开源库,所以功能强大但代码量极少,仅600+行,大致架构如下: [image.png
name: additional-scrape-configs 然后将原有的scrape configs配置保存到prometheus-additional.yaml文件中,如: - job_name: 'kafka-exporter
17.53MiB / 47.01GiB 0.04% 10.9GB / 37GB 0B / 0B 4 afe6d4ebe409 kafka-exporter
17.53MiB / 47.01GiB 0.04% 10.9GB / 37GB 0B / 0B 4 afe6d4ebe409 kafka-exporter
17.53MiB / 47.01GiB 0.04% 10.9GB / 37GB 0B / 0B 4 afe6d4ebe409 kafka-exporter
17.53MiB / 47.01GiB 0.04% 10.9GB / 37GB 0B / 0B 4 afe6d4ebe409 kafka-exporter