Kafka producer在创建Debezium MySQL连接器后无法创建主题并抛出连续错误

Question

我使用Debezium作为CDC工具来流式传输来自MySql的数据。在将Debezium连接器安装到Confluent OSS集群后，我正在尝试将MySQL bin_log的变化捕获到Kafka主题中。当我创建连接器时，在获取数据库的快照之后，我会遇到一系列连续的错误。

我检查了MySql bin_log是否打开，并尝试使用不同的序列化程序重新启动模式注册表和连接器。但是我得到了同样的错误。

错误日志显示：

[2019-06-21 13:56:14,885] INFO Step 8: - Completed scanning a total of 955 rows from table 'mydb.test' after 00:00:00.086 (io.debezium.connector.mysql.SnapshotReader:565)
[2019-06-21 13:56:14,886] INFO Step 8: scanned 1758 rows in 2 tables in 00:00:00.383 (io.debezium.connector.mysql.SnapshotReader:601)
[2019-06-21 13:56:14,886] INFO Step 9: committing transaction (io.debezium.connector.mysql.SnapshotReader:635)
[2019-06-21 13:56:14,887] INFO Completed snapshot in 00:00:01.055 (io.debezium.connector.mysql.SnapshotReader:701)
[2019-06-21 13:56:14,965] WARN [Producer clientId=producer-5] Error while fetching metadata with correlation id 11 : {kbserver=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient:968)
[2019-06-21 13:56:15,066] WARN [Producer clientId=producer-5] Error while fetching metadata with correlation id 12 : {kbserver=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient:968)
[2019-06-21 13:56:15,168] WARN [Producer clientId=producer-5] Error while fetching metadata with correlation id 13 : {kbserver=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient:968)
[2019-06-21 13:56:15,269] WARN [Producer clientId=producer-5] Error while fetching metadata with correlation id 14 : {kbserver=UNKNOWN_TOPIC_OR_PARTITION} (org.apache.kafka.clients.NetworkClient:968)
[2019-06-21 13:56:15,370] WARNDebezium [Producer clientId=producer-5] Error while fetching metadata with correlation id 15 : {kbserver=UNKNOWN_TOPIC_OR_PARTITION}

我发送的连接器负载如下所示：

{
      "name": "debezium-connector",
      "config": {
            "connector.class": "io.debezium.connector.mysql.MySqlConnector",
            "tasks.max": "1",
            "key.serializer": "io.confluent.connect.avro.AvroConverter",
            "value.serializer": "io.confluent.connect.avro.AvroConverter",
            "database.hostname": "localhost",
            "database.port": "3306",
            "database.user": "test",
            "database.password": "test@123",
            "database.whitelist": "mydb",
            "table.whitelist": "mydb.test",
            "database.server.id": "1",
            "database.server.name": "kbserver",
            "database.history.kafka.bootstrap.servers": "kafka:9092",
            "database.history.kafka.topic": "db-schema.mydb",
            "include.schema.changes": "true"
       }
    }

是否有人知道为什么会发生这种情况，或者我如何修复它？

Answer 1

请参考database.whitelist和table.whitelist。它们是不一致的。

它应该是mydb和mydb.test，或者是db和db.test，具体取决于数据库名称。