PostgreSQL到Druid摄入投掷NullPointerException

Question

我正在尝试从PostgreSQL到Druid摄取数据。创建索引任务并抛出NullPointerExcpetion失败。

摄食规格

{
    "type": "index",
    "spec": {
        "dataSchema": {
            "dataSource": "dataset_new10"
        },
        "ioConfig": {
            "type": "index",
            "firehose": {
                "type": "sql",
                "database": {
                    "type": "postgresql",
                    "connectorConfig": {
                        "connectURI": "jdbc:postgresql://ISVDRDBILXP1/testdb",
                        "user": "druid",
                        "password": "druid"
                    }
                },
                "sqls": [
                    "SELECT * FROM public.testtable"
                ]
            }
        }
    }
}

在这里我的堆叠痕迹：

2019-07-10T12:59:22,853 INFO [task-runner-0-priority-0] org.apache.druid.data.input.impl.prefetch.CacheManager - Object[SELECT * FROM public.testtable] is cached. Current cached bytes is [188]
2019-07-10T12:59:22,858 ERROR [task-runner-0-priority-0] org.apache.druid.indexing.common.task.IndexTask - Encountered exception in DETERMINE_PARTITIONS.
java.lang.NullPointerException
    at org.apache.druid.data.input.impl.SqlFirehose.nextRow(SqlFirehose.java:68) ~[druid-core-0.15.0-incubating.jar:0.15.0-incubating]
    at org.apache.druid.indexing.common.task.IndexTask.collectIntervalsAndShardSpecs(IndexTask.java:744) ~[druid-indexing-service-0.15.0-incubating.jar:0.15.0-incubating]
    at org.apache.druid.indexing.common.task.IndexTask.createShardSpecsFromInput(IndexTask.java:671) ~[druid-indexing-service-0.15.0-incubating.jar:0.15.0-incubating]
    at org.apache.druid.indexing.common.task.IndexTask.determineShardSpecs(IndexTask.java:606) ~[druid-indexing-service-0.15.0-incubating.jar:0.15.0-incubating]
    at org.apache.druid.indexing.common.task.IndexTask.run(IndexTask.java:437) [druid-indexing-service-0.15.0-incubating.jar:0.15.0-incubating]
    at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:419) [druid-indexing-service-0.15.0-incubating.jar:0.15.0-incubating]
    at org.apache.druid.indexing.overlord.SingleTaskBackgroundRunner$SingleTaskBackgroundRunnerCallable.call(SingleTaskBackgroundRunner.java:391) [druid-indexing-service-0.15.0-incubating.jar:0.15.0-incubating]
    at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_212]
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_212]
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_212]
    at java.lang.Thread.run(Thread.java:748) [?:1.8.0_212]

Answer 1

对于Druid0.20.1，我修复了这个错误，添加了一些JSON标记：

{
  "type": "index_parallel",
  "spec": {
    "dataSchema": {
      "dataSource": "test-ingestion-postgresql",
      "timestampSpec": {
        "format": "iso",
        "column": "ts",
        "missingValue": "2021-05-05T00:00:00.000"
      },
      "dimensionsSpec": {
        "dimensions": [
          "column1",
          "column2",
          "column3"
        ]
      }
    },
    "ioConfig": {
      "type": "index_parallel",
      "inputSource": {
        "type": "sql",
        "database": {
          "type": "postgresql",
          "connectorConfig": {
            "connectURI": "jdbc:postgresql://host:5432/db_name",
            "user": "user",
            "password": "password"
          }
        },
        "sqls": [
          "SELECT column1, column2, column3 FROM schema.table"
        ]
      }
    },
    "tuningConfig": {
      "type": "index_parallel",
      "partitionsSpec": {
        "type": "dynamic"
      }
    }
  }
}