从Python写入在Docker中运行的Kafka失败

Question

我使用Docker Toolbox和Docker Compose在OS X上的Docker中运行了Zookeeper和Kafka。我的docker-compose.yml文件是：

zk:
  image: patrickmay/zookeeper:3.4.6-jessie
  ports:
   - "2181:2181"
  command: /usr/local/zookeeper/bin/zkServer.sh start-foreground
kafka0:
  image: patrickmay/kafka:0.8.2.1-jessie
  ports:
   - "9092:9092"
  command: /usr/local/bin/start-kafka.sh 0 docker-machine 9092
  links:
   - zk
kafka1:
  image: patrickmay/kafka:0.8.2.1-jessie
  ports:
   - "9093:9092"
  command: /usr/local/bin/start-kafka.sh 1 docker-machine 9093
  links:
   - zk
kafka2:
  image: patrickmay/kafka:0.8.2.1-jessie
  ports:
   - "9094:9092"
  command: /usr/local/bin/start-kafka.sh 2 docker-machine 9094
  links:
   - zk

我创建了一个名为test_impressions的主题：

~/packages/kafka_2.11-0.8.2.1/bin/kafka-topics.sh --zookeeper docker-machine --describe --topic test_impressions
Topic:test_impressions  PartitionCount:3    ReplicationFactor:2 Configs:
    Topic: test_impressions Partition: 0    Leader: 0   Replicas: 2,0   Isr: 0
    Topic: test_impressions Partition: 1    Leader: 1   Replicas: 0,1   Isr: 1
    Topic: test_impressions Partition: 2    Leader: 1   Replicas: 1,2   Isr: 1

我试着用这个简单的Python脚本来写它：

#!/usr//bin/env python

from __future__ import print_function
import sys
from kafka import SimpleProducer, KafkaClient

kafka = KafkaClient('docker-machine:9094')
producer = SimpleProducer(kafka)

producer.send_messages(b'test_impressions', b'some message')

我得到了这个错误：

Traceback (most recent call last):
  File "./kafka-producer.py", line 12, in <module>
    producer.send_messages(b'test_impressions', b'some message')
  File "/Users/patrick.may/projects/ad-server/frequency-cap/venv/lib/python2.7/site-packages/kafka/producer/simple.py", line 52, in send_messages
    partition = self._next_partition(topic)
  File "/Users/patrick.may/projects/ad-server/frequency-cap/venv/lib/python2.7/site-packages/kafka/producer/simple.py", line 36, in _next_partition
    self.client.load_metadata_for_topics(topic)
  File "/Users/patrick.may/projects/ad-server/frequency-cap/venv/lib/python2.7/site-packages/kafka/client.py", line 378, in load_metadata_for_topics
    kafka.common.check_error(topic_metadata)
  File "/Users/patrick.may/projects/ad-server/frequency-cap/venv/lib/python2.7/site-packages/kafka/common.py", line 233, in check_error
    raise error_class(response)
kafka.common.LeaderNotAvailableError:     TopicMetadata(topic='test_impressions', error=5, partitions=[])

我已经尝试了所有三个对接机端口(9092、9093和9094)。我有什么配置错误吗？

谢谢。

Answer 1

找到了我的错误。在docker-compose.yml文件中使用docker-machine是错误的，因为docker-machine不在/etc/hosts文件中。将其更改为数字IP地址解决了此问题。