如何解决Neo4J-admin导入中的此未排序数据错误

Question

经过一些试验和错误，我已经设法将我的数据强制到一组最初被Neo4j-admin import接受的CSV文件中。但是，在导入最大文件的大约70%时，程序崩溃并出现Unsorted data, a > b Failure错误。我还没有找到任何关于需要对数据进行排序的内容，如果是这样，我不确定应该对哪些列进行排序。我清除了我的数据中的任何NaN值，以防这是问题所在，但它并没有解决它。

下面是完整的日志，如果有人有任何想法的话。谢谢!

bash-3.2$ bin/neo4j-admin import --id-type INTEGER --nodes:User "import/user_nodes.csv" --nodes:Step "import/step_nodes.csv" --relationships:IS_ROOT_OF "import/root_edges.csv" --relationships:COLLECTED_AT_STEP "import/step_edges.csv" --max-memory=8g
Neo4j version: 3.5.8
Importing the contents of these files into /Users/James/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-fc317459-d0c3-43c0-b1eb-d16df6b0d172/installation-3.5.8/data/databases/graph.db:
Nodes:
  :User
  /Users/James/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-fc317459-d0c3-43c0-b1eb-d16df6b0d172/installation-3.5.8/import/user_nodes.csv

  :Step
  /Users/James/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-fc317459-d0c3-43c0-b1eb-d16df6b0d172/installation-3.5.8/import/step_nodes.csv
Relationships:
  :IS_ROOT_OF
  /Users/James/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-fc317459-d0c3-43c0-b1eb-d16df6b0d172/installation-3.5.8/import/root_edges.csv

  :COLLECTED_AT_STEP
  /Users/James/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-fc317459-d0c3-43c0-b1eb-d16df6b0d172/installation-3.5.8/import/step_edges.csv

Available resources:
  Total machine memory: 16.00 GB
  Free machine memory: 3.45 GB
  Max heap memory : 4.44 GB
  Processors: 8
  Configured max memory: 10.40 GB
  High-IO: true

Import starting 2019-09-17 13:24:26.679+0100
  Estimated number of nodes: 11.60 M
  Estimated number of node properties: 92.81 M
  Estimated number of relationships: 25.60 M
  Estimated number of relationship properties: 0.00 
  Estimated disk space usage: 2.27 GB
  Estimated required memory usage: 1.14 GB

InteractiveReporterInteractions command list (end with ENTER):
  c: Print more detailed information about current stage
  i: Print more detailed information

(1/4) Node import 2019-09-17 13:24:26.905+0100
  Estimated number of nodes: 11.60 M
  Estimated disk space usage: 1.46 GB
  Estimated required memory usage: 1.14 GB
.......... .......... .......... .......... ..........   5% ∆3s 933ms
.......... .......... .......... .......... ..........  10% ∆2s 816ms
.......... .......... .......... .......... ..........  15% ∆2s 412ms
.......... .......... .......... .......... ..........  20% ∆3s 31ms
.......... .......... .......... .......... ..........  25% ∆2s 227ms
.......... .......... .......... .......... ..........  30% ∆2s 624ms
.......... ....-..... .......... .......... ..........  35% ∆754ms
.......... .......... .......... .......... ..........  40% ∆1ms
.......... .......... .......... .......... ..........  45% ∆0ms
.......... .......... .......... .......... ..........  50% ∆1s 406ms
.......... .......... .......... .......... ..........  55% ∆1s 5ms
.......... .......... .......... .......... ..........  60% ∆2s 822ms
.......... .......... .......... .......... ..........  65% ∆3s 215ms
.......... .......... .......... .......... .........Exception in thread "DETECT-6" java.lang.RuntimeException: java.lang.IllegalStateException: Unsorted data, a > b Failure:[10526719] 2efffffbfcd78001 > 2cc08cb655d4d000 | 4194302:3154733
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers$Worker.run(Workers.java:131)
Caused by: java.lang.IllegalStateException: Unsorted data, a > b Failure:[10526719] 2efffffbfcd78001 > 2cc08cb655d4d000 | 4194302:3154733
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.detect(EncodingIdMapper.java:367)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.run(EncodingIdMapper.java:335)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers$Worker.run(Workers.java:126)
Exception in thread "DETECT-5" java.lang.RuntimeException: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers$Worker.run(Workers.java:131)
Caused by: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.detect(EncodingIdMapper.java:367)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.run(EncodingIdMapper.java:335)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers$Worker.run(Workers.java:126)
Exception in thread "Thread-5" java.lang.RuntimeException: java.lang.RuntimeException: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
        at org.neo4j.unsafe.impl.batchimport.staging.AbstractStep.issuePanic(AbstractStep.java:155)
        at org.neo4j.unsafe.impl.batchimport.staging.AbstractStep.issuePanic(AbstractStep.java:147)
        at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:59)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.RuntimeException: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers.awaitAndThrowOnError(Workers.java:82)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers.awaitAndThrowOnErrorStrict(Workers.java:90)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.detectAndMarkCollisions(EncodingIdMapper.java:442)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.prepare(EncodingIdMapper.java:236)
        at org.neo4j.unsafe.impl.batchimport.IdMapperPreparationStep.process(IdMapperPreparationStep.java:56)
        at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:53)
        ... 1 more
Caused by: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.detect(EncodingIdMapper.java:367)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.run(EncodingIdMapper.java:335)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers$Worker.run(Workers.java:126)
.  70% ∆1s 791ms
.......... .......... .......... .......... ..........  75% ∆0ms
.......... .......... .......... .......... ..........  80% ∆1ms
.......... .......... .......... .......... ..........  85% ∆0ms
.......... .......... .......... .......... ..........  90% ∆0ms
.......... .......... .......... .......... ..........  95% ∆0ms
.......... .......... .......... .......... .......... 100% ∆1ms


IMPORT FAILED in 29s 410ms. 
Data statistics is not available.
Peak memory usage: 1.09 GB
Import error: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
Caused by:java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
java.lang.RuntimeException: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers.awaitAndThrowOnError(Workers.java:82)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers.awaitAndThrowOnErrorStrict(Workers.java:90)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.detectAndMarkCollisions(EncodingIdMapper.java:442)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper.prepare(EncodingIdMapper.java:236)
        at org.neo4j.unsafe.impl.batchimport.IdMapperPreparationStep.process(IdMapperPreparationStep.java:56)
        at org.neo4j.unsafe.impl.batchimport.staging.LonelyProcessingStep.lambda$receive$0(LonelyProcessingStep.java:53)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.detect(EncodingIdMapper.java:367)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.EncodingIdMapper$DetectWorker.run(EncodingIdMapper.java:335)
        at org.neo4j.unsafe.impl.batchimport.cache.idmapping.string.Workers$Worker.run(Workers.java:126)

WARNING Import failed. The store files in /Users/James/Library/Application Support/Neo4j Desktop/Application/neo4jDatabases/database-fc317459-d0c3-43c0-b1eb-d16df6b0d172/installation-3.5.8/data/databases/graph.db are left as they are, although they are likely in an unusable state. Starting a database on these store files will likely fail or observe inconsistent records so start at your own risk or delete the store manually
unexpected error: java.lang.IllegalStateException: Unsorted data, a > b Failure:[8948597] 363e6cefb256e003 > 2c000001e9960000 | 1022779:0
bash-3.2$ 

Answer 1

在感觉到我的ID字段有问题后，我将原始ID字段复制到一个新列中，以便在需要将记录与原始源进行匹配时保存，并创建了一个新的ID列，该列是一个简单的整数范围。

这解决了问题，但出现了一个新的问题，我认为这解释了最初的问题。我新创建的原始ID字段抛出了一个错误，因为数字太长而不是整数。我将此列从整型重新分配为长整型，问题已修复。因此，我的理论是，用于ID的原始值太长，无法存储为整数。