将数据大容量加载到Cassandra中的NoHostAvailableException
将数据大容量加载到Cassandra中的NoHostAvailableException
提问于 2014-03-25 11:43:51
我正试着把100万个LUBM数据集加载到卡桑德拉。但是，在大约20万次插入之后，我仍然得到了这个异常。
线程“主”com.datastax.driver.core.exceptions.NoHostAvailableException:中的异常所有主机尝试查询失败(尝试: localhost/127.0.0.1在读取期间(com.datastax.driver.core.exceptions.DriverException:超时)
服务器还活着。我仍然可以使用执行CQL查询。在此之前，我得到了WriteTimeoutException，因此我在.yaml中调整了一些选项。
以下是我的系统规格：
CPU-英特尔的I5 -2核心-内存- 4GB的硬盘- 500 GB 4驱动器。
我有一个OWL文件，从中读取数据并插入Cassandra。我的工作量是混合的。有时，根据某个条件，我需要从该行中检索数据，并将新数据附加到该行并将其写入同一行。这不一定是造成这个问题的原因，因为这种情况在200 k插入过程中一定发生过很多次。
我使用外部硬盘来存储提交日志。concurrent_reads设置为16，concurrent_writes设置为32。read_timeout被设置为10000，与write_timeout一样。MAX_HEAP_SIZE="2G“HEAP_NEWSIZE=200米
一旦我得到帽子异常，我就会看到这个模式的日志。
INFO [MemoryMeter:1] 2014-03-25 16:28:56,498 Memtable.java (line 449) CFS(Keyspace='rdf', ColumnFamily='lubm1m.lubm1m_teacherof_idx') liveRatio is 15.733535462912831 (just-counted was 15.733535462912831). calculation took 91ms for 665 cells
INFO [MemoryMeter:1] 2014-03-25 16:30:01,560 Memtable.java (line 449) CFS(Keyspace='system', ColumnFamily='sstable_activity') liveRatio is 15.896188158961882 (just-counted was 15.896188158961882). calculation took 37ms for 411 cells
INFO [ScheduledTasks:1] 2014-03-25 16:33:22,252 GCInspector.java (line 116) GC for ParNew: 15581 ms for 1 collections, 395151608 used; max is 1038876672
INFO [ScheduledTasks:1] 2014-03-25 16:33:24,894 StatusLogger.java (line 55) Pool Name Active Pending Completed Blocked All Time Blocked
INFO [ScheduledTasks:1] 2014-03-25 16:33:26,956 StatusLogger.java (line 70) ReadStage 0 0 343094 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:26,957 StatusLogger.java (line 70) RequestResponseStage 0 0 1 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:26,996 StatusLogger.java (line 70) ReadRepairStage 0 0 0 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,004 StatusLogger.java (line 70) MutationStage 0 0 601747 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,024 StatusLogger.java (line 70) ReplicateOnWriteStage 0 0 0 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,031 StatusLogger.java (line 70) GossipStage 0 0 0 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,066 StatusLogger.java (line 70) AntiEntropyStage 0 0 0 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,067 StatusLogger.java (line 70) MigrationStage 0 0 0 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,098 StatusLogger.java (line 70) MemtablePostFlusher 0 0 52 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,099 StatusLogger.java (line 70) MemoryMeter 0 0 139 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,100 StatusLogger.java (line 70) FlushWriter 0 0 31 0 16
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,101 StatusLogger.java (line 70) MiscStage 0 0 0 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,130 StatusLogger.java (line 70) PendingRangeCalculator 0 0 1 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,133 StatusLogger.java (line 70) commitlog_archiver 0 0 0 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,134 StatusLogger.java (line 70) InternalResponseStage 0 0 0 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,142 StatusLogger.java (line 70) HintedHandoff 0 0 0 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,223 StatusLogger.java (line 79) CompactionManager 0 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,232 StatusLogger.java (line 81) Commitlog n/a 0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,262 StatusLogger.java (line 93) MessagingService n/a 0/0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,262 StatusLogger.java (line 103) Cache Type Size Capacity KeysToSave
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,287 StatusLogger.java (line 105) KeyCache 17212 51380224 all
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,335 StatusLogger.java (line 111) RowCache 0 0 all
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,336 StatusLogger.java (line 118) ColumnFamily Memtable ops,data
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,336 StatusLogger.java (line 121) system.schema_triggers 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,337 StatusLogger.java (line 121) system.local 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,337 StatusLogger.java (line 121) system.peers 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,338 StatusLogger.java (line 121) system.batchlog 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,338 StatusLogger.java (line 121) system.NodeIdInfo 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,339 StatusLogger.java (line 121) system.compactions_in_progress 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,339 StatusLogger.java (line 121) system.schema_keyspaces 4,1130
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,340 StatusLogger.java (line 121) system.compaction_history 38,11220
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,340 StatusLogger.java (line 121) system.paxos 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,340 StatusLogger.java (line 121) system.schema_columns 3361,891818
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,341 StatusLogger.java (line 121) system.schema_columnfamilies 1426,394018
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,341 StatusLogger.java (line 121) system.IndexInfo 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,342 StatusLogger.java (line 121) system.range_xfers 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,344 StatusLogger.java (line 121) system.peer_events 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,344 StatusLogger.java (line 121) system.hints 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,345 StatusLogger.java (line 121) system.sstable_activity 1542,196000
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,345 StatusLogger.java (line 121) KeyspaceCumulus.POS_RN_NUM 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,346 StatusLogger.java (line 121) KeyspaceCumulus.DICT_SO_REVERSE 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,346 StatusLogger.java (line 121) KeyspaceCumulus.SPO_RN_DT 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,347 StatusLogger.java (line 121) KeyspaceCumulus.DICT_P_REVERSE 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,347 StatusLogger.java (line 121) KeyspaceCumulus.SPOC 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,348 StatusLogger.java (line 121) KeyspaceCumulus.POSC 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,348 StatusLogger.java (line 121) KeyspaceCumulus.counter 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,349 StatusLogger.java (line 121) KeyspaceCumulus.SCHEMA_O_PROPS 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,350 StatusLogger.java (line 121) KeyspaceCumulus.SCHEMA_D_PROPS 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,350 StatusLogger.java (line 121) KeyspaceCumulus.PREFIX_TO_NS 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,352 StatusLogger.java (line 121) KeyspaceCumulus.POS_RN_DT 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,352 StatusLogger.java (line 121) KeyspaceCumulus.SPO_RN_NUM 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,353 StatusLogger.java (line 121) KeyspaceCumulus.SCHEMA_CLASSES 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,353 StatusLogger.java (line 121) KeyspaceCumulus.OSPC 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,354 StatusLogger.java (line 121) KeyspaceCumulus.DICT_SO 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,354 StatusLogger.java (line 121) KeyspaceCumulus.DICT_P 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,355 StatusLogger.java (line 121) rdf.animal 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,355 StatusLogger.java (line 121) rdf.lubm20 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,356 StatusLogger.java (line 121) rdf.bsbm50 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,356 StatusLogger.java (line 121) rdf.tim 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,357 StatusLogger.java (line 121) rdf.bsbm250 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,357 StatusLogger.java (line 121) rdf.lubm1m 152700,22140112
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,358 StatusLogger.java (line 121) rdf.lubm3 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,358 StatusLogger.java (line 121) system_traces.sessions 0,0
INFO [ScheduledTasks:1] 2014-03-25 16:33:27,359 StatusLogger.java (line 121) system_traces.events 0,0有什么问题吗？
回答 1
Stack Overflow用户
发布于 2014-03-26 08:00:01
使用SocketOptions增加套接字超时。您也可以尝试在datastax客户端和服务器端配置它。另外，看看降低一致性有什么帮助吗？
-Vivek
页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/22633566
复制相关文章
相似问题
腾讯云开发者
Copyright © 2013 - 2026 Tencent Cloud. All Rights Reserved. 腾讯云 版权所有
深圳市腾讯计算机系统有限公司 ICP备案/许可证号：粤B2-20090059 
粤公网安备44030502008569号
腾讯云计算（北京）有限责任公司 京ICP证150476号 | 京ICP备11018762号