在docker-desktop k8s集群上运行strimzi时找不到Kafka主题

Question

我尝试在this document之后，在docker-desktop k8s上运行strimzi。

我把所有的东西都安排好了。

$ kubectl get all -n my-kafka-project                                           
NAME                                              READY   STATUS    RESTARTS   AGE
pod/my-cluster-entity-operator-7ddb6d5b88-q2xg8   3/3     Running   0          97m
pod/my-cluster-kafka-0                            1/1     Running   0          98m
pod/my-cluster-zookeeper-0                        1/1     Running   0          98m

NAME                                          TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                      AGE
service/my-cluster-kafka-0                    NodePort    10.99.207.179    <none>        9094:31805/TCP               98m
service/my-cluster-kafka-bootstrap            ClusterIP   10.103.193.53    <none>        9091/TCP,9092/TCP,9093/TCP   98m
service/my-cluster-kafka-brokers              ClusterIP   None             <none>        9091/TCP,9092/TCP,9093/TCP   98m
service/my-cluster-kafka-external-bootstrap   NodePort    10.97.198.62     <none>        9094:31314/TCP               98m
service/my-cluster-zookeeper-client           ClusterIP   10.101.206.203   <none>        2181/TCP                     98m
service/my-cluster-zookeeper-nodes            ClusterIP   None             <none>        2181/TCP,2888/TCP,3888/TCP   98m

NAME                                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/my-cluster-entity-operator   1/1     1            1           97m

NAME                                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/my-cluster-entity-operator-7ddb6d5b88   1         1         1       97m

NAME                                    READY   AGE
statefulset.apps/my-cluster-kafka       1/1     98m
statefulset.apps/my-cluster-zookeeper   1/1     98m

并获取nodeport：

$ kubectl get service my-cluster-kafka-external-bootstrap -n my-kafka-project -o=jsonpath='{.spec.ports[0].nodePort}{"\n"}'
31314

虽然本文使用的是localhost:31314，但我可以通过localhost访问docker-desktop节点，所以我尝试访问minikube，但无法向topic生成消息。似乎我可以成功连接到broker，但不能连接到topic。

./kafka-console-producer.sh --broker-list localhost:31314 --topic my-topic 
>Test
[2021-05-15 15:41:13,681] ERROR Error when sending message to topic my-topic with key: null, value: 2 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Topic my-topic not present in metadata after 60000 ms.

我已经和kafkacat确认过了，topic本身创建成功。

kafkacat -b localhost:31314 -L
Metadata for all topics (from broker -1: localhost:31314/bootstrap):
 1 brokers:
  broker 0 at 192.168.65.4:31805 (controller)
 4 topics:
  topic "__strimzi_store_topic" with 1 partitions:
    partition 0, leader 0, replicas: 0, isrs: 0
  topic "my-topic" with 3 partitions:
    partition 0, leader 0, replicas: 0, isrs: 0
    partition 1, leader 0, replicas: 0, isrs: 0
    partition 2, leader 0, replicas: 0, isrs: 0
  topic "__strimzi-topic-operator-kstreams-topic-store-changelog" with 1 partitions:
    partition 0, leader 0, replicas: 0, isrs: 0
  topic "__consumer_offsets" with 50 partitions:
    partition 0, leader 0, replicas: 0, isrs: 0
    partition 1, leader 0, replicas: 0, isrs: 0
    partition 2, leader 0, replicas: 0, isrs: 0
    partition 3, leader 0, replicas: 0, isrs: 0
    partition 4, leader 0, replicas: 0, isrs: 0
    partition 5, leader 0, replicas: 0, isrs: 0
    ...

我不完全理解消息是如何发送到topic的机制，那么下一步应该检查哪一点？

Answer 1

根据Jakub的注释，我可以通过改变my-kafka-project命名空间中的Kafka资源来产生和消费来自主机的消息。

我已经覆盖了.spec.kafka.listeners的外部侦听器中通告的侦听器。

发自：

- name: external
  port: 9094
  tls: false
  type: nodeport

至：

- name: external
  port: 9094
  tls: false
  type: nodeport
  configuration:
    brokers:
    - broker: 0
      advertisedHost: localhost
      advertisedPort: 31314