牧场桌面上的纤毛装置

Question

我不太清楚如何正确地表达这个问题，真的是k8s的初学者。我正在为笔记本电脑上的k3s创建一个操场，希望安装纤毛和prometheus/grafana监控。为此，我安装了牧场主桌面，它创建了在vm中运行k3s的沙箱环境(运行在笔记本电脑上)。

我在牧场主桌面v1.0.0上安装了纤毛，并安装了头盔：

helm install cilium cilium/cilium --version 1.11.1 \
   --namespace kube-system \
   --set prometheus.enabled=true \
   --set operator.prometheus.enabled=true \
   --set hubble.enabled=true \
   --set hubble.metrics.enabled="{dns,drop,tcp,flow,port-distribution,icmp,http}"

它安装，但纤毛容器未能启动：

Error: failed to generate container "0fae98546697febc25abb4ac49d5e5a2f27a3ee1781bade900f2c767f8d6df28" spec: failed to generate spec: path "/run/cilium/cgroupv2" is mounted on "/run/cilium/cgroupv2" but it is not a shared or slave mount

这使我相信，bpf没有安装。现在，我对Linux很熟悉，但我真的是k8s的新手。牧场主桌面封装k3s并在vm中启动它(我的pc是基于Ubuntu20.04的)。因此，qemu启动这个vm (利马-牧场主-桌面)，我可以登录到它。我假设BPF应该在那个VM中启用(它不是)。但是也许我错了。也许是在k3s里面的一些容器里？我甚至在我的笔记本电脑上启用了BPF，但是这并没有帮助，k3s运行在那个VM中，下面是吊舱：

kubectl get pods -A
NAMESPACE           NAME                                      READY   STATUS                      RESTARTS   AGE
kube-system         local-path-provisioner-84bb864455-z2659   1/1     Running                     0          4h28m
kube-system         helm-install-traefik-crd--1-dxcg7         0/1     Completed                   0          4h28m
kube-system         svclb-traefik-7kqgd                       2/2     Running                     0          4h28m
kube-system         helm-install-traefik--1-lbjhw             0/1     Completed                   1          4h28m
kube-system         metrics-server-ff9dbcb6c-rmvd9            1/1     Running                     0          4h28m
kube-system         traefik-55fdc6d984-zpk5s                  1/1     Running                     0          4h28m
cilium-monitoring   prometheus-655fb888d7-mbnb9               1/1     Running                     0          3h52m
cilium-monitoring   grafana-5747bcc8f9-rj5jk                  1/1     Running                     0          3h52m
kube-system         cilium-operator-5ffd7d9795-ktldm          0/1     Pending                     0          3m26s
kube-system         cilium-operator-5ffd7d9795-b8ls9          1/1     Running                     0          3m26s
kube-system         cilium-d5xr4                              0/1     Init:CreateContainerError   0          3m26s
kube-system         coredns-96cc4f57d-r99zl                   1/1     Running                     0          7s

如果有人能解释一下应该在哪里安装BPF，我会很感激:在这个VM内部，或者在k3s上的某个容器中，以及如何安装它？

注意:无论如何，它不会在VM中挂载：

mount --bind /var/run/bpf /var/run/bpf

没有任何效果，也不会挂载任何东西，好像它是只读的。

Answer 1

在四处游玩之后，我发现它必须安装在VM上而不是主机上。如果BPF FS已经挂载，但没有共享，则必须将其卸载，然后再次以共享方式挂载：

sudo mount bpffs -t bpf /sys/fs/bpf
sudo mount --make-shared /sys/fs/bpf

Answer 2

基于上述发现的完整解决方案：https://github.com/cilium/cilium/issues/18675#issuecomment-1050234756

摘要：

好的，所以我自己安装组并让它共享是很容易的。

因此，总共：

sudo mount bpffs -t bpf /sys/fs/bpf
sudo mount --make-shared /sys/fs/bpf
sudo mkdir -p /run/cilium/cgroupv2
sudo mount -t cgroup2 none /run/cilium/cgroupv2
sudo mount --make-shared /run/cilium/cgroupv2/

在客户机内部，我创建了一个脚本，我只是通过limactl调用它(主机上的$HOME被挂载到来宾中)：

(⎈ |rancher-desktop:default) ~/g/s/g/c/cilium ❯❯❯ cat setup-cilium-rancher.sh                                                                                                                 fix_grep ◼
#!/bin/sh

set -e

echo Mounting bpf
mount bpffs -t bpf /sys/fs/bpf
mount --make-shared /sys/fs/bpf

echo Mounting cgroups v2 to /run/cilium/cgroupv2
mkdir -p /run/cilium/cgroupv2
mount -t cgroup2 none /run/cilium/cgroupv2
mount --make-shared /run/cilium/cgroupv2/
(⎈ |rancher-desktop:default) ~/g/s/g/c/cilium ❯❯❯ LIMA_HOME="$HOME/Library/Application Support/rancher-desktop/lima" "/Applications/Rancher Desktop.app/Contents/Resources/resources/darwin/lima/bin/limactl" shell 0 sudo sh $PWD/setup-cilium-rancher.sh
Mounting bpf
Mounting cgroups v2 to /run/cilium/cgroupv2

然后我做了一个基本的舵机安装，我们没有问题地运行纤毛DaemonSet吊舱：

(⎈ |rancher-desktop:default) ~/g/s/g/c/cilium ❯❯❯ helm install cilium cilium/cilium --version 1.11.2 --namespace kube-system                                                                  fix_grep ◼
W0224 12:24:45.168567   78675 warnings.go:70] spec.template.spec.affinity.nodeAffinity.requiredDuringSchedulingIgnoredDuringExecution.nodeSelectorTerms[1].matchExpressions[0].key: beta.kubernetes.io/os is deprecated since v1.14; use "kubernetes.io/os" instead
W0224 12:24:45.168579   78675 warnings.go:70] spec.template.metadata.annotations[scheduler.alpha.kubernetes.io/critical-pod]: non-functional in v1.16+; use the "priorityClassName" field instead
NAME: cilium
LAST DEPLOYED: Thu Feb 24 12:24:44 2022
NAMESPACE: kube-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
You have successfully installed Cilium with Hubble.

Your release version is 1.11.2.

For any further help, visit https://docs.cilium.io/en/v1.11/gettinghelp

(⎈ |rancher-desktop:default) ~/g/s/g/c/cilium ❯❯❯ kubectl get pods -n kube-system                                                                                                                   fix_grep ◼
NAME                                      READY   STATUS      RESTARTS   AGE
local-path-provisioner-84bb864455-57c46   1/1     Running     0          6m20s
helm-install-traefik-crd--1-jdxsr         0/1     Completed   0          6m21s
metrics-server-ff9dbcb6c-ft2c4            1/1     Running     0          6m20s
helm-install-traefik--1-nflmd             0/1     Completed   2          6m21s
svclb-traefik-tgr4h                       2/2     Running     0          6m
traefik-55fdc6d984-l2skq                  1/1     Running     0          6m
cilium-operator-6d8799bcbb-f74g8          0/1     Pending     0          3m15s
cilium-operator-6d8799bcbb-njzk2          1/1     Running     0          3m15s
cilium-h7qzw                              1/1     Running     0          3m15s
coredns-96cc4f57d-zkjlg                   1/1     Running     0          12s