带minikube和metallb的nginx

Question

您好，我正尝试在minikube中使用我自己的容器启动我自己的部署。这是我的yaml文件

apiVersion: apps/v1
kind: Deployment
metadata:
  name: wildboar-nginx-depl
  labels:
    app: services.nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: services.nginx
  template:
    metadata:
      labels:
        app: services.nginx
    spec:
      containers:
        - name: wildboar-nginx-pod
          image: services.nginx
          ports:
            - containerPort: 80
            - containerPort: 443
            - containerPort: 22
          imagePullPolicy: Never
---
apiVersion: v1
kind: Service
metadata:
  name: wildboar-nginx-service
  annotations: 
    metallb.universe.tf/allow-shared-ip: wildboar-key
spec:
  type: LoadBalancer
  loadBalancerIP: 192.168.1.101 
  selector:
    app: services.nginx
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
      nodePort: 30080
    - name: https
      protocol: TCP
      port: 443
      targetPort: 443
      nodePort: 30443
    - name: ssh
      protocol: TCP
      port: 22
      targetPort: 22
      nodePort: 30022

那是我的Dockerfile

FROM alpine:latest
RUN apk update && apk upgrade -U -a
RUN apk add nginx openssl openrc openssh supervisor
RUN mkdir /www/
RUN adduser -D -g 'www' www
RUN chown -R www:www /www
RUN chown -R www:www /var/lib/nginx
RUN openssl req -x509 -nodes -days 30 -newkey rsa:2048 -subj \
"/C=RU/ST=Moscow/L=Moscow/O=lchantel/CN=localhost" -keyout \
/etc/ssl/private/lchantel.key -out /etc/ssl/certs/lchantel.crt
COPY ./conf /etc/nginx/conf.d/default.conf
COPY ./nginx_conf.sh .
COPY ./supervisor.conf /etc/
RUN mkdir -p /run/nginx/
EXPOSE 80 443 22
RUN chmod 755 /nginx_conf.sh
CMD sh nginx_conf.sh

那是我的nginx_conf.sh

#!bin/sh

cp /var/lib/nginx/html/index.html /www/
rc default
rc-service sshd start
ssh-keygen -A
rc-service sshd stop
/usr/bin/supervisord -c /etc/supervisord.conf

在我成功地实现了yaml文件之后，但是我被CrashLoopBackOff错误卡住了：

$ kubectl get pod
NAME                                   READY   STATUS             RESTARTS   AGE
wildboar-nginx-depl-57d64f58d8-cwcnn   0/1     CrashLoopBackOff   2          40s
wildboar-nginx-depl-57d64f58d8-swmq2   0/1     CrashLoopBackOff   2          40s

我尝试重新启动，但没有帮助。我试图描述pod，但信息没有帮助：

$ kubectl describe pod wildboar-nginx-depl-57d64f58d8-cwcnn
Name:         wildboar-nginx-depl-57d64f58d8-cwcnn
Namespace:    default
Priority:     0
Node:         minikube/192.168.99.100
Start Time:   Sun, 06 Dec 2020 17:49:19 +0300
Labels:       app=services.nginx
              pod-template-hash=57d64f58d8
Annotations:  <none>
Status:       Running
IP:           172.17.0.7
IPs:
  IP:           172.17.0.7
Controlled By:  ReplicaSet/wildboar-nginx-depl-57d64f58d8
Containers:
  wildboar-nginx-pod:
    Container ID:   docker://6bd4ab3b08703293697d401e355d74d1ab09f938eb23b335c92ffbd2f8f26706
    Image:          services.nginx
    Image ID:       docker://sha256:a62f240db119e727935f072686797f5e129ca44cd1a5f950e5cf606c9c7510b8
    Ports:          80/TCP, 443/TCP, 22/TCP
    Host Ports:     0/TCP, 0/TCP, 0/TCP
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Sun, 06 Dec 2020 17:52:13 +0300
      Finished:     Sun, 06 Dec 2020 17:52:15 +0300
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Sun, 06 Dec 2020 17:50:51 +0300
      Finished:     Sun, 06 Dec 2020 17:50:53 +0300
    Ready:          False
    Restart Count:  5
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-hr82j (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  default-token-hr82j:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-hr82j
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason     Age                  From               Message
  ----     ------     ----                 ----               -------
  Normal   Scheduled  3m9s                                    Successfully assigned default/wildboar-nginx-depl-57d64f58d8-cwcnn to minikube
  Normal   Pulled     98s (x5 over 3m9s)   kubelet, minikube  Container image "services.nginx" already present on machine
  Normal   Created    98s (x5 over 3m9s)   kubelet, minikube  Created container wildboar-nginx-pod
  Normal   Started    98s (x5 over 3m9s)   kubelet, minikube  Started container wildboar-nginx-pod
  Warning  BackOff    59s (x10 over 3m4s)  kubelet, minikube  Back-off restarting failed container

我没有主意了，我该怎么办呢？

Answer 1

我用nginx解决了这个问题。首先，我重写了supervisor.conf，它现在是这样的：

[supervisord]
nodaemon=true
user = root

[program:nginx]
command=nginx -g 'daemon off;'
autostart=true
autorestart=true
startsecs=0
redirect_stderr=true

[program:ssh]
command=/usr/sbin/sshd -D
autostart=true
autorestart=true

第二，我对loadBalancer有意见。我在文件中交换了服务和部署配置，还为服务添加了下一个统计信息spec.externalTrafficPolicy: Cluster (用于ip地址共享)。

apiVersion: v1
kind: Service
metadata:
  name: wildboar-nginx-service
  labels:
    app: nginx
  annotations: 
    metallb.universe.tf/allow-shared-ip: minikube
spec:
  type: LoadBalancer
  loadBalancerIP: 192.168.99.105
  selector:
    app: nginx
  externalTrafficPolicy: Cluster
  ports:
    - name: http
      protocol: TCP
      port: 80
      targetPort: 80
    - name: https
      protocol: TCP
      port: 443
      targetPort: 443
    - name: ssh
      protocol: TCP
      port: 22
      targetPort: 22

---

apiVersion: apps/v1
kind: Deployment
metadata:
  name: wildboar-nginx-depl
  labels:
    app: nginx
spec:
  replicas: 1
  strategy:
    type: Recreate
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
     restartPolicy: Always
     containers:
       - name: wildboar-nginx-pod
         image: wildboar.nginx:latest
         ports:
           - containerPort: 80
             name: http
           - containerPort: 443
             name: https
           - containerPort: 22
             name: ssh
         imagePullPolicy: Never

第三，我用如下脚本重新构建了minikube和所有的配置

#!/bin/bash

kubectl ns default
kubectl delete deployment --all
kubectl delete service --all
kubectl ns metallb-system
kubectl delete configmap --all
kubectl ns default
docker rmi -f <your_custom_docker_image>
minikube stop
minikube delete 
minikube start --driver=virtualbox --disk-size='<your size>mb' --memory='<your_size>mb'
minikube addons enable metallb
eval $(minikube docker-env)
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/namespace.yaml
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.9.5/manifests/metallb.yaml
# next line is only when you use mettallb for first time
#kubectl create secret generic -n metallb-system memberlist --from-literal=secretkey="$(openssl rand -base64 128)"
docker build -t <your_custom_docker_images> .
kubectl apply -f <mettalb_yaml_config>.yaml
kubectl apply -f <your_config_with_deployment_and_service>.yaml

我还提到，yaml文件对空格和制表符非常敏感，所以我安装了yamllint来对yaml文件进行基本调试。我要感谢迷茫的天才和David Maze的帮助！