dockerd在重新启动时未重新启动，并显示消息"failed to start containerd: timeout waiting for containerd to start“

Question

这里是第一篇文章，所以为任何错误道歉。

我有一个docker环境，它展示了一个非常奇怪的问题。当我在18.09.2版本时，它可以完美地工作，但后来我需要升级docker版本，因为一些docker需要它，因为API版本( IIRC )发生了变化。

我已经升级到20.10.2 (没有重启)，一切似乎都很好，dockers启动了，我可以使用它们了。一段时间后，我的电源故障导致我重新启动，从那时起，我就有了这个问题。在启动时dockers命令会导致：

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

因此，我在日志( /var/log/docker.log )中搜索并找到：

time="2021-08-30T16:40:11.702266553+02:00" level=info msg="Starting up"
time="2021-08-30T16:40:11.715505120+02:00" level=warning msg="could not change group /var/run/docker.sock to docker: group docker not found"
time="2021-08-30T16:40:11.728188524+02:00" level=info msg="libcontainerd: started new containerd process" pid=9883
time="2021-08-30T16:40:11.728497763+02:00" level=info msg="parsed scheme: \"unix\"" module=grpc
time="2021-08-30T16:40:11.728564781+02:00" level=info msg="scheme \"unix\" not registered, fallback to default scheme" module=grpc
time="2021-08-30T16:40:11.728723243+02:00" level=info msg="ccResolverWrapper: sending update to cc: {[{unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}] <nil> <nil>}" module=grpc
time="2021-08-30T16:40:11.728841483+02:00" level=info msg="ClientConn switching balancer to \"pick_first\"" module=grpc
time="2021-08-30T16:40:11.813209337+02:00" level=info msg="starting containerd" revision=269548fa27e0089a8b8278fc4fc781d7f65a939b version=1.4.3
time="2021-08-30T16:40:11.928783093+02:00" level=info msg="loading plugin \"io.containerd.content.v1.content\"..." type=io.containerd.content.v1
time="2021-08-30T16:40:11.929009055+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..." type=io.containerd.snapshotter.v1
time="2021-08-30T16:40:11.936721860+02:00" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exit status 1 \"modprobe: FATAL: Modu
le aufs not found in directory /lib/modules/5.4.65-v7l-sarpi4\\n\"): skip plugin" type=io.containerd.snapshotter.v1
time="2021-08-30T16:40:11.936880396+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." type=io.containerd.snapshotter.v1
time="2021-08-30T16:40:11.937437133+02:00" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.btrfs (ext4) 
must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2021-08-30T16:40:11.937510744+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.devmapper\"..." type=io.containerd.snapshotter.v1
time="2021-08-30T16:40:11.937618391+02:00" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
time="2021-08-30T16:40:11.937684465+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.native\"..." type=io.containerd.snapshotter.v1
time="2021-08-30T16:40:11.937796094+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.overlayfs\"..." type=io.containerd.snapshotter.v1
time="2021-08-30T16:40:11.938041796+02:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.zfs\"..." type=io.containerd.snapshotter.v1
time="2021-08-30T16:40:11.938477682+02:00" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.zfs\"..." error="path /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.zfs must be a z
fs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2021-08-30T16:40:11.938549200+02:00" level=info msg="loading plugin \"io.containerd.metadata.v1.bolt\"..." type=io.containerd.metadata.v1
time="2021-08-30T16:40:11.938622793+02:00" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
time="2021-08-30T16:40:11.938674255+02:00" level=info msg="metadata content store policy set" policy=shared
time="2021-08-30T16:40:11.938972068+02:00" level=info msg="loading plugin \"io.containerd.differ.v1.walking\"..." type=io.containerd.differ.v1
time="2021-08-30T16:40:11.939055994+02:00" level=info msg="loading plugin \"io.containerd.gc.v1.scheduler\"..." type=io.containerd.gc.v1
time="2021-08-30T16:40:11.939191530+02:00" level=info msg="loading plugin \"io.containerd.service.v1.introspection-service\"..." type=io.containerd.service.v1
time="2021-08-30T16:40:11.939374825+02:00" level=info msg="loading plugin \"io.containerd.service.v1.containers-service\"..." type=io.containerd.service.v1
time="2021-08-30T16:40:11.939489232+02:00" level=info msg="loading plugin \"io.containerd.service.v1.content-service\"..." type=io.containerd.service.v1
time="2021-08-30T16:40:11.939557250+02:00" level=info msg="loading plugin \"io.containerd.service.v1.diff-service\"..." type=io.containerd.service.v1
time="2021-08-30T16:40:11.939634268+02:00" level=info msg="loading plugin \"io.containerd.service.v1.images-service\"..." type=io.containerd.service.v1
time="2021-08-30T16:40:11.939699008+02:00" level=info msg="loading plugin \"io.containerd.service.v1.leases-service\"..." type=io.containerd.service.v1
time="2021-08-30T16:40:11.939768008+02:00" level=info msg="loading plugin \"io.containerd.service.v1.namespaces-service\"..." type=io.containerd.service.v1
time="2021-08-30T16:40:11.939834674+02:00" level=info msg="loading plugin \"io.containerd.service.v1.snapshots-service\"..." type=io.containerd.service.v1
time="2021-08-30T16:40:11.939925785+02:00" level=info msg="loading plugin \"io.containerd.runtime.v1.linux\"..." type=io.containerd.runtime.v1
time="2021-08-30T16:40:11.940284968+02:00" level=info msg="loading plugin \"io.containerd.runtime.v2.task\"..." type=io.containerd.runtime.v2
time="2021-08-30T16:40:12.729504178+02:00" level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc 
= \"transport: error while dialing: dial unix:///var/run/docker/containerd/containerd.sock: timeout\". Reconnecting..." module=grpc
time="2021-08-30T16:40:15.081866772+02:00" level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc 
= \"transport: error while dialing: dial unix:///var/run/docker/containerd/containerd.sock: timeout\". Reconnecting..." module=grpc
time="2021-08-30T16:40:18.723223037+02:00" level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc 
= \"transport: error while dialing: dial unix:///var/run/docker/containerd/containerd.sock: timeout\". Reconnecting..." module=grpc
time="2021-08-30T16:40:23.950263284+02:00" level=warning msg="grpc: addrConn.createTransport failed to connect to {unix:///var/run/docker/containerd/containerd.sock  <nil> 0 <nil>}. Err :connection error: desc 
= \"transport: error while dialing: dial unix:///var/run/docker/containerd/containerd.sock: timeout\". Reconnecting..." module=grpc
failed to start containerd: timeout waiting for containerd to start

我把头撞到墙上，最后我发现如果我把

/var/run/docker/containerd 

目录我可以毫无问题地启动dockerd和containerd，但显然会丢失每个docker实例，并且需要docker rm和docker重新启动我的容器。

你知道为什么会发生这种情况吗？

我的环境：

root@casa:/var/adm/packages# cat /etc/slackware-version 
Slackware 14.2+
root@casa:/var/adm/packages# uname -a
Linux casa.pigi.org 5.4.65-v7l-sarpi4 #3 SMP Mon Sep 21 10:13:26 BST 2020 armv7l BCM2711 GNU/Linux
root@casa:/var/adm/packages# docker info
Client:
 Context:    default
 Debug Mode: false

Server:
 Containers: 5
  Running: 5
  Paused: 0
  Stopped: 0
 Images: 9
 Server Version: 20.10.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc version: 
 init version: fec3683 (expected: de40ad0)
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.4.65-v7l-sarpi4
 Operating System: Slackware 14.2 arm (post 14.2 -current)
 OSType: linux
 Architecture: armv7l
 CPUs: 4
 Total Memory: 3.738GiB
 Name: casa.pigi.org
 ID: HF4Y:7TDZ:O5GV:HM7H:YCVS:CLKW:GNOM:6PSA:XRCQ:3BQU:TZ3P:URLD
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Username: pigi102
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

WARNING: No swap limit support
WARNING: No blkio weight support
WARNING: No blkio weight_device support

root@casa:/var/adm/packages# runc -v
runc version spec: 1.0.1-dev

root@casa:/var/adm/packages# /usr/bin/docker -v
Docker version 20.10.2, build 2291f61

root@casa:/var/adm/packages# containerd -v
containerd github.com/containerd/containerd 1.4.3 269548fa27e0089a8b8278fc4fc781d7f65a939b

docker-proxy-20201215_fa125a3

提前谢谢。

Pigi_102

Answer 1

我做了更多的测试，似乎如果我运行containerd (像dockerd一样使用所有的选项和标志)并等待足够长的时间，它最终会启动，并且dockerd可以从那里启动。

Answer 2

我设法解决了我的问题，通过降级到docker 19.03.15和containerd 1.2.13，在这些版本中，一切都像预期的那样工作。

Pigi