启动过程中遇到的cassandra异常: index (1)必须小于size (1)

Question

我经历了一次停电，杀死了卡桑德拉星系团的所有4个节点。我已经把所有的盒子都带回来了，但是现在当我尝试启动Cassandra (方法bin/Cassandra)时，我在4个中的3个上得到了下面的消息，在这里有一些回放日志。

Exception encountered during startup: index (1) must be less than size (1)
 INFO 10:11:00,479 CFS(Keyspace='system', ColumnFamily='peers') liveRatio is 13.10204081632653 (just-counted was 13.10204081632653).  calculation took 18ms for 21 columns

(到此为止，如果有帮助的话，我可以发布更多的信息)

出现的是只运行读，我认为，因为我有复制因子3在所有的键空间，所以它不能工作，没有其他。

任何想法都不知道该怎么办才能挽回局面。我在数据库里有很多有价值的信息。

卡桑德拉诉1.6，

CentOs v6.4

更新：

在重放提交日志时，所有三个死节点似乎都失败了。我已经把他们接到的线路插上了线。我最初的想法是删除提交日志并尝试重新启动。但我有点害怕后果！！

 INFO 10:46:48,600 Replaying /var/lib/cassandra/commitlog/CommitLog-2-1379365208291.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208292.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208293.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208294.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208295.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208296.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208297.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208298.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208299.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208301.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208302.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208304.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208305.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208306.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208307.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208308.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208309.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208310.log, /var/lib/cassandra/commitlog/CommitLog-2-1379365208311.log, /var/lib/cassandra/commitlog/CommitLog-2-1381251642075.log, /var/lib/cassandra/commitlog/CommitLog-2-1381251731119.log
 INFO 10:46:48,614 Replaying /var/lib/cassandra/commitlog/CommitLog-2-1379365208291.log
 INFO 10:46:49,598 GC for ParNew: 605 ms for 2 collections, 50055912 used; max is 1046937600
 INFO 10:46:51,775 Finished reading /var/lib/cassandra/commitlog/CommitLog-2-1379365208291.log
 INFO 10:46:51,776 Replaying /var/lib/cassandra/commitlog/CommitLog-2-1379365208292.log
 INFO 10:46:53,995 Finished reading /var/lib/cassandra/commitlog/CommitLog-2-1379365208292.log
 INFO 10:46:53,995 Replaying /var/lib/cassandra/commitlog/CommitLog-2-1379365208293.log
 **INFO 10:46:54,087 Finished reading /var/lib/cassandra/commitlog/CommitLog-2-1379365208293.log**
ERROR 10:46:54,088 Exception encountered during startup
java.lang.IndexOutOfBoundsException: index (1) must be less than size (1)
    at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:305)
    at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:284)
    at com.google.common.collect.SingletonImmutableList.get(SingletonImmutableList.java:45)
    at org.apache.cassandra.db.marshal.CompositeType.getComparator(CompositeType.java:94)
    at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:76)
    at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:31)
    at java.util.TreeMap.compare(TreeMap.java:1188)
    at java.util.TreeMap.put(TreeMap.java:531)
    at org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:102)
    at org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:88)
    at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:114)
    at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:109)
    at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:101)
    at org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:397)
    at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:202)
    at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:97)
    at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:146)
    at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:126)
    at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:298)
    at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:441)
    at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:484)
java.lang.IndexOutOfBoundsException: index (1) must be less than size (1)
    at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:305)
    at com.google.common.base.Preconditions.checkElementIndex(Preconditions.java:284)
    at com.google.common.collect.SingletonImmutableList.get(SingletonImmutableList.java:45)
    at org.apache.cassandra.db.marshal.CompositeType.getComparator(CompositeType.java:94)
    at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:76)
    at org.apache.cassandra.db.marshal.AbstractCompositeType.compare(AbstractCompositeType.java:31)
    at java.util.TreeMap.compare(TreeMap.java:1188)
    at java.util.TreeMap.put(TreeMap.java:531)
    at org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:102)
    at org.apache.cassandra.db.TreeMapBackedSortedColumns.addColumn(TreeMapBackedSortedColumns.java:88)
    at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:114)
    at org.apache.cassandra.db.AbstractColumnContainer.addColumn(AbstractColumnContainer.java:109)
    at org.apache.cassandra.db.ColumnFamilySerializer.deserialize(ColumnFamilySerializer.java:101)
    at org.apache.cassandra.db.RowMutation$RowMutationSerializer.deserialize(RowMutation.java:397)
    at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:202)
    at org.apache.cassandra.db.commitlog.CommitLogReplayer.recover(CommitLogReplayer.java:97)
    at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:146)
    at org.apache.cassandra.db.commitlog.CommitLog.recover(CommitLog.java:126)
    at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:298)
    at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:441)
    at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:484)
Exception encountered during startup: index (1) must be less than size (1)
 INFO 10:46:54,110 CFS(Keyspace='system', ColumnFamily='peers') liveRatio is 13.10204081632653 (just-counted was 13.10204081632653).  calculation took 19ms for 21 columns

Answer 1

我最好的理论是，这可能是删除和重新创建同名表(https://issues.apache.org/jira/browse/CASSANDRA-5905)的结果。我们的目标是2.1 (https://issues.apache.org/jira/browse/CASSANDRA-5202)的修复；同时，我们更喜欢截断而不是drop/recreate。

Answer 2

虽然已经很晚了，你也必须解决这个问题。对于有类似问题的其他人，请尝试以下方法

sudo bash -c 'rm -rf /var/lib/cassandra/data/system/*'

记住，这只是为了发展的目的。此命令将删除所有数据.