在运行近2分钟后，dsbulk卸载在特定行上失败。

Question

我们希望将Cassandra表中的所有数据导出到CSV中。

dsbulk unload -k KEYSPACE -t TABLE -f ./application.conf > export_TABLE.csv

如果这样做，它将在1分50秒后以1次错误完成，输出如下：

total     | failed | rows/s | p50ms |  p99ms | p999ms
2,082,040 |      1 | 18,858 | 99.03 | 224.40 | 278.92

每次都在同一位置失败。

所以我的问题是，我如何确定错误出现在哪里？

拥有更多数据的Update：

这是"unload_errors.log“文件输出：

Statement: com.datastax.oss.driver.internal.core.cql.DefaultBoundStatement@30e63f93 [2 values, idempotence: , CL: , serial CL: , timestamp: , timeout: ]
SELECT column1, column2, column3, column4, column5, column6, column7 FROM keyspace.table WHERE token(column1) > :start AND token(column1) <= :end
start: -291276829359193218
end: 282879262594721796
com.datastax.oss.dsbulk.executor.api.exception.BulkExecutionException: Statement execution failed: SELECT column1, column2, column3, column4, column5, column6, column7 FROM keyspace.table WHERE token(column1) > :start AND token(column1) <= :end (Cassandra timeout during read query at consistency LOCAL_ONE (1 responses were required but only 0 replica responded))
        at com.datastax.oss.dsbulk.executor.api.subscription.ResultSubscription.toErrorPage(ResultSubscription.java:534)
        at com.datastax.oss.dsbulk.executor.api.subscription.ResultSubscription.lambda$fetchNextPage$1(ResultSubscription.java:372)
        at com.datastax.oss.driver.internal.core.cql.CqlRequestHandler.setFinalError(CqlRequestHandler.java:447) [4 skipped]
        at com.datastax.oss.driver.internal.core.cql.CqlRequestHandler.access$700(CqlRequestHandler.java:94)
        at com.datastax.oss.driver.internal.core.cql.CqlRequestHandler$NodeResponseCallback.processRetryVerdict(CqlRequestHandler.java:859)
        at com.datastax.oss.driver.internal.core.cql.CqlRequestHandler$NodeResponseCallback.processErrorResponse(CqlRequestHandler.java:828)
        at com.datastax.oss.driver.internal.core.cql.CqlRequestHandler$NodeResponseCallback.onResponse(CqlRequestHandler.java:655)
        at com.datastax.oss.driver.internal.core.channel.InFlightHandler.channelRead(InFlightHandler.java:257)
        at java.base/java.lang.Thread.run(Thread.java:830) [24 skipped]
Caused by: com.datastax.oss.driver.api.core.servererrors.ReadTimeoutException: Cassandra timeout during read query at consistency LOCAL_ONE (1 responses were required but only 0 replica responded)

注:卡桑德拉正在这台机器上的一个码头集装箱中运行。

“operation.log”的输出：

2022-08-09 06:45:48 INFO  Operation directory: /root/logs/UNLOAD_20220809-064548-285327
2022-08-09 06:45:49 WARN  [driver] localhost/127.0.0.1:9042 did not send an authentication challenge; This is suspicious because the driver expects authentication
2022-08-09 06:45:49 WARN  [driver] localhost/127.0.0.1:9042 did not send an authentication challenge; This is suspicious because the driver expects authentication
2022-08-09 06:45:49 WARN  [driver] localhost/127.0.0.1:9042 did not send an authentication challenge; This is suspicious because the driver expects authentication
2022-08-09 06:45:49 WARN  [driver] localhost/127.0.0.1:9042 did not send an authentication challenge; This is suspicious because the driver expects authentication
2022-08-09 06:45:49 WARN  [driver] localhost/127.0.0.1:9042 did not send an authentication challenge; This is suspicious because the driver expects authentication
2022-08-09 06:45:49 WARN  [driver] localhost/127.0.0.1:9042 did not send an authentication challenge; This is suspicious because the driver expects authentication
2022-08-09 06:45:49 WARN  [driver] localhost/127.0.0.1:9042 did not send an authentication challenge; This is suspicious because the driver expects authentication
2022-08-09 06:45:49 WARN  [driver] localhost/127.0.0.1:9042 did not send an authentication challenge; This is suspicious because the driver expects authentication
2022-08-09 06:45:49 WARN  [driver] localhost/127.0.0.1:9042 did not send an authentication challenge; This is suspicious because the driver expects authentication
2022-08-09 06:47:40 WARN  Operation UNLOAD_20220809-064548-285327 completed with 1 errors in 1 minute and 50 seconds.
2022-08-09 06:47:40 INFO  Records: total: 2,082,040, successful: 2,082,039, failed: 1
2022-08-09 06:47:40 INFO  Memory usage: used: 392 MB, free: 963 MB, allocated: 1,356 MB, available: 3,908 MB, total gc count: 34, total gc time: 873 ms
2022-08-09 06:47:40 INFO  Reads: total: 2,082,040, successful: 2,082,039, failed: 1, in-flight: 0
2022-08-09 06:47:40 INFO  Throughput: 18,853 reads/second
2022-08-09 06:47:40 INFO  Latencies: mean 94.82, 75p 113.25, 99p 216.01, 999p 331.35 milliseconds
2022-08-09 06:47:42 INFO  Final stats:
2022-08-09 06:47:42 INFO  Records: total: 2,082,040, successful: 2,082,039, failed: 1
2022-08-09 06:47:42 INFO  Memory usage: used: 393 MB, free: 962 MB, allocated: 1,356 MB, available: 3,908 MB, total gc count: 34, total gc time: 873 ms
2022-08-09 06:47:42 INFO  Reads: total: 2,082,040, successful: 2,082,039, failed: 1, in-flight: 0
2022-08-09 06:47:42 INFO  Throughput: 18,505 reads/second
2022-08-09 06:47:42 INFO  Latencies: mean 94.82, 75p 113.25, 99p 216.01, 999p 331.35 milliseconds

输出：

Table: TABLENAME
SSTable count: 5
Space used (live): 420485304
Space used (total): 420485304
Space used by snapshots (total): 0
Off heap memory used (total): 79392
SSTable Compression Ratio: 0.6476976670697281
Number of partitions (estimate): 24
Memtable cell count: 5
Memtable data size: 8
Memtable off heap memory used: 0
Memtable switch count: 0
Local read count: 15
Local read latency: NaN ms
Local write count: 5
Local write latency: NaN ms
Pending flushes: 0
Percent repaired: 0.0
Bloom filter false positives: 0
Bloom filter false ratio: 0.00000
Bloom filter space used: 128
Bloom filter off heap memory used: 88
Index summary off heap memory used: 80
Compression metadata off heap memory used: 79224
Compacted partition minimum bytes: 18
Compacted partition maximum bytes: 322381140
Compacted partition mean bytes: 14284800
Average live cells per slice (last five minutes): 5389.228571428572
Maximum live cells per slice (last five minutes): 5722
Average tombstones per slice (last five minutes): 1.0555555555555556
Maximum tombstones per slice (last five minutes): 2
Dropped Mutations: 0

Answer 1

当DSBulk遇到问题时，它会报告指向STDOUT的日志的路径，包括一个错误文件。

检查输出文件，特别是错误日志，因为它们将为导出失败提供线索。干杯!