Ubuntu与systemd -如何得到一个回溯或核心？

Question

我使用的是Ubuntu18.04服务器和systemd。最近，我的部门开发的一个程序在一天内两次崩溃，错误如下：

Jun 07 06:33:07 xxx systemd[1]: xxx.service: Main process exited, code=killed, status=11/SEGV
Jun 07 06:33:07 xxx systemd[1]: xxx.service: Failed with result 'signal'.

我想下一步是得到一个回溯或coredump，但是我不知道如何在使用systemd的Ubuntu上做到这一点。

我不确定是否应该继续使用systemd-coredump、coredumpctl或其他实用程序。

而且，我也不知道该发出什么命令。对于上面的实用程序，有很多关于各种特性的文档，等等，但是我找不到一个简洁的例子，比如：

sudo apt-get install xyz

(run x, y, z commands to get core dump)

有人能提供一个简洁的例子或教程网站来解释这一点吗？我不需要也不想使用各种复杂的特性，我只是想得到一个基本的核心转储。

Answer 1

例如，一个相对简单的服务，example守护进程。

使用dbgsym包安装调试符号.不幸的是，默认情况下，ddebs不在源文件中。另外，没有很好的脚本可以找到包，所以从将-dbgsym附加到包名开始。

sudo apt install chrony-dbgsym

您可能需要考虑如何处理现代Linux服务器上的核心转储.和他们的例子中，他们正在考虑回到仅仅核心转储文件。就我个人而言，我在服务器上的apport中没有任何有用的东西，但是我发现coredumpctl是有用的。因此，对Ubuntu 18.04的系统方法：

sudo systemctl stop apport
sudo systemctl mask --now apport
sudo apt install systemd-coredump
# Verify this changed the core pattern to a pipe to systemd-coredump
sysctl kernel.core_pattern

# Reproduce the crash.
sudo killall -s SIGSEGV chronyd

# List collected dumps.
coredumpctl

# Invoke debugger on the latest one.
sudo coredumpctl gdb
# systemd >= 239  the gdb verb was renamed debug. Also, select core by PID.
sudo coredumpctl debug 5809

# In GDB, the basic thing to get is a stack trace. Ask the developer what else they want.
(gdb) thread apply all bt

启动调试器会话可能如下所示：

John@coredump:~$ coredumpctl
TIME                            PID   UID   GID SIG COREFILE  EXE
Sat 2019-06-08 12:55:16 UTC    5809   111   115  11 error     /usr/sbin/chronyd
John@coredump:~$ sudo coredumpctl gdb
           PID: 5809 (chronyd)
           UID: 111 (_chrony)
           GID: 115 (_chrony)
        Signal: 11 (SEGV)
     Timestamp: Sat 2019-06-08 12:55:16 UTC (1h 19min ago)
  Command Line: /usr/sbin/chronyd
    Executable: /usr/sbin/chronyd
 Control Group: /system.slice/chrony.service
          Unit: chrony.service
         Slice: system.slice
       Boot ID: c9a0a69a73d245c1ae5dfe7d491ead0a
    Machine ID: d2934a6e67f81ae0097be31003da0b31
      Hostname: coredump
       Storage: /var/lib/systemd/coredump/core.chronyd.111.c9a0a69a73d245c1ae5dfe7d491ead0a.5809.1559998516000000.lz4
       Message: Process 5809 (chronyd) of user 111 dumped core.

                Stack trace of thread 5809:
                #0  0x00007eff1ce1403f __GI___select (libc.so.6)
                #1  0x00005597867eb3be n/a (chronyd)
                #2  0x00005597867e1071 n/a (chronyd)
                #3  0x00007eff1cd1eb97 __libc_start_main (libc.so.6)
                #4  0x00005597867e127a n/a (chronyd)

GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /usr/sbin/chronyd...Reading symbols from /usr/lib/debug/.build-id/89/dcd398c87777f4c869bfd0831215eeb8b6c7fe.debug...done.
done.
[New LWP 5809]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `/usr/sbin/chronyd'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007eff1ce1403f in __GI___select (nfds=4, readfds=readfds@entry=0x7ffc0fd73c80,
    writefds=writefds@entry=0x0, exceptfds=exceptfds@entry=0x0, timeout=timeout@entry=0x7ffc0fd73be0)
    at ../sysdeps/unix/sysv/linux/select.c:41
41      ../sysdeps/unix/sysv/linux/select.c: No such file or directory.
(gdb) thread apply all bt

Thread 1 (Thread 0x7eff1df14740 (LWP 5809)):
#0  0x00007eff1ce1403f in __GI___select (nfds=4, readfds=readfds@entry=0x7ffc0fd73c80,
    writefds=writefds@entry=0x0, exceptfds=exceptfds@entry=0x0, timeout=timeout@entry=0x7ffc0fd73be0)
    at ../sysdeps/unix/sysv/linux/select.c:41
#1  0x00005597867eb3be in SCH_MainLoop () at sched.c:747
#2  0x00005597867e1071 in main (argc=, argv=0x7ffc0fd73fb8) at main.c:605

在这个精心设计的例子中，在select()中捕获了它，因为我粗鲁地向等待I/O的任务发送了一个信号。

更复杂的软件可能缺少其他符号，安装这些符号或者源代码并继续调试。