Prometheus服务因无效参数错误而失败

Question

Prometheus在这个错误中失败了：

Aug 04 11:27:36 rockpi prometheus[12548]: level=info ts=2020-08-04T11:27:36.098Z caller=main.go:549 msg="Scrape discovery manager stopped"
Aug 04 11:27:36 rockpi prometheus[12548]: level=info ts=2020-08-04T11:27:36.098Z caller=manager.go:888 component="rule manager" msg="Stopping rule manager..."
Aug 04 11:27:36 rockpi prometheus[12548]: level=info ts=2020-08-04T11:27:36.098Z caller=main.go:583 msg="Scrape manager stopped"
Aug 04 11:27:36 rockpi prometheus[12548]: level=info ts=2020-08-04T11:27:36.098Z caller=manager.go:898 component="rule manager" msg="Rule manager stopped"
Aug 04 11:27:36 rockpi prometheus[12548]: level=info ts=2020-08-04T11:27:36.099Z caller=main.go:563 msg="Notify discovery manager stopped"
Aug 04 11:27:36 rockpi prometheus[12548]: level=info ts=2020-08-04T11:27:36.099Z caller=notifier.go:601 component=notifier msg="Stopping notification manager..."
Aug 04 11:27:36 rockpi prometheus[12548]: level=info ts=2020-08-04T11:27:36.099Z caller=main.go:755 msg="Notifier manager stopped"
Aug 04 11:27:36 rockpi prometheus[12548]: level=error ts=2020-08-04T11:27:36.100Z caller=main.go:764 err="opening storage failed: mmap files, file: /var/lib/prometheus/chunks_head/000022: mmap: invalid argument"
Aug 04 11:27:36 rockpi systemd[1]: prometheus.service: Main process exited, code=exited, status=1/FAILURE
Aug 04 11:27:36 rockpi systemd[1]: prometheus.service: Failed with result 'exit-code'.

所以我发现下面的错误是由什么引起的呢？

 --storage.tsdb.path /var/lib/prometheus/

完整的命令是：

$ prometheus --config.file /etc/prometheus/prometheus.yml --storage.tsdb.path /var/lib/prometheus/ --web.console.templates=/etc/prometheus/consoles

结果此错误：

panic: Unable to create mmap-ed active query log

goroutine 1 [running]:
github.com/prometheus/prometheus/promql.NewActiveQueryTracker(0x7fe5c48690, 0x14, 0x14, 0x2964a00, 0x400087dc20, 0x2964a00)
    /app/promql/query_logger.go:117 +0x38c
main.main()
    /app/cmd/prometheus/main.go:374 +0x44c8

我该如何解决这个问题？

Answer 1

在我的例子中，Prometheus是作为服务运行的，而systemctrl无法启动它。我从树中删除了整个目录(以防万一)。我是这样做的：

prometheus[12528]: level=error ts=2021-09-20T06:13:16.727Z caller=main.go:798 err="opening storage failed: mmap files, file: /opt/prometheus/chunks_head/001064: mmap: invalid argument" 

这是可行的。

Answer 2

我也遇到了同样的问题，我只是清空了chunks_head

sudo rm -rf /var/lib/prometheus/chunks_head/

在那之后，服务似乎启动正常，数据看起来也正常。不过，我不确定这会有什么影响。