无法使用坞文件连接到坞中的kafka。

Question

version: '2'
services:
  zookeeper:
    image: confluentinc/cp-zookeeper:5.5.1
    hostname: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    volumes:
      - ./.docker-volumes/zookeeper/_data:/var/lib/zookeeper/data
      - ./.docker-volumes/zookeeper/_log:/var/lib/zookeeper/log

  kafka:
    image: confluentinc/cp-kafka:5.5.1
    depends_on:
      - zookeeper
    ports:
      - 9092:9092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:29092,PLAINTEXT_HOST://0.0.0.0:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    volumes:
      - ./.docker-volumes/kafka/_data:/var/lib/kafka/data

以上是码头-合成文件，以旋转动物园管理员和卡夫卡实例在码头。

当我运行这个服务的时候，它试图连接到卡夫卡。我的错误越来越少。我在互联网上找到了它，但在这方面没有得到多少。

服务层上的错误

.12:43:18.730 [kafka-producer-network-thread | producer-2] WARN  o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-2] Connection to node -1 (localhost/127.0.0.1:9092) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. 
.12:43:19.295 [kafka-producer-network-thread | producer-1] WARN  o.apache.kafka.clients.NetworkClient - [Producer clientId=producer-1] Connection to node -1 (localhost/127.0.0.1:9092) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. 
.12:43:19.481 [kafka-admin-client-thread | vendor_srs_consumer_srs_restaurant_finance_pan_validate_client_id_0] WARN  o.apache.kafka.clients.NetworkClient - [AdminClient clientId=vendor_srs_consumer_srs_restaurant_finance_pan_validate_client_id_0] Connection to node -1 (localhost/127.0.0.1:9092) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue. 

当卡夫卡-制片人试图与卡夫卡经纪人联系时，码头上的错误。

[2021-07-12 07:13:19,492] WARN [SocketServer brokerId=1] Unexpected error from /172.20.0.1; closing connection (org.apache.kafka.common.network.Selector)
org.apache.kafka.common.network.InvalidReceiveException: Invalid receive (size = 369296129 larger than 104857600)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:105)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:447)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:397)
at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:678)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:580)
at org.apache.kafka.common.network.Selector.poll(Selector.java:485)
at kafka.network.Processor.poll(SocketServer.scala:893)
at kafka.network.Processor.run(SocketServer.scala:792)
at java.lang.Thread.run(Thread.java:748)

不确定为什么生产者不能与代理连接。提前谢谢。

Answer 1

PLAINTEXT_HOST://0.0.0.0:9092不是一个有效的公告侦听器；这需要是一个可解析的地址。

在您的错误中，Connection to node -1 (localhost/127.0.0.1:9092) terminated，不清楚这段代码在哪里运行，但是如果在Docker之外的同一主机上，那么您应该宣传localhost，而不是0.0.0.0

另一个错误是，InvalidReceiveException: Invalid receive (size = 369296129 larger than 104857600)表示您已连接到其他服务器，并得到了客户端意想不到的响应。

Answer 2

最初，它们处于默认的桥接网络中，为了在容器之间进行通信，您必须使用容器IP。

您可以尝试在同一个网络上部署两个容器。

在停靠-组合中添加一个网络定义

networks:
  default:
    external:
      name: zookeeper-kafka
  zookeeper-kafka:
    driver: bridge

在服务定义中，可以添加网络名称和端口。

例如，动物园管理员服务

zookeeper:
    networks:
    - zookeeper-kafka
    image: confluentinc/cp-zookeeper:5.5.1
    hostname: zookeeper
    container_name: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    volumes:
      - ./.docker-volumes/zookeeper/_data:/var/lib/zookeeper/data
      - ./.docker-volumes/zookeeper/_log:/var/lib/zookeeper/log

然后您可以通过动物园管理员- kafka :2181(即网络名称:端口)访问动物园管理员服务，同时通过kafka进行连接。

更新kafka服务定义示例

kafka:
    networks:
    - zookeeper-kafka
    image: confluentinc/cp-kafka:5.5.1
    depends_on:
      - zookeeper
    ports:
      - 9092:9092
    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper-kafka:2181
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://zookeeper-kafka:29092,PLAINTEXT_HOST://0.0.0.0:9092
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    volumes:
      - ./.docker-volumes/kafka/_data:/var/lib/kafka/data

希望这能帮到你