使用mapdb时内存不足

Question

测试mapdb时出现内存不足错误。考虑到该项目的整个想法是将数据结构序列化到磁盘并避免内存问题，我认为我做错了什么。你知道我做错了什么吗？或者是有bug？

    @Test
    public void testLarge() throws Exception {
        final HTreeMap<UUID, String> storage = DBMaker.newTempHashMap();

        String string = createDataSize(250);

        ArrayList<UUID> keys = new ArrayList<>();
        for (int i = 0; i < 320000; i++) {
            final UUID key = UUID.randomUUID();
            storage.put(key, string);
            keys.add(key);
        }

        for (UUID key : keys) {
            assertNotNull(storage.get(key));
        }

        for (UUID key : keys) {
            storage.remove(key);
        }

        assertEquals("nothing left", 0, storage.size());
    }

    /**
     * Creates a message of size @msgSize in KB.
     */
    private static String createDataSize(int msgSize) {
        // Java chars are 2 bytes
        msgSize = msgSize / 2;
        msgSize = msgSize * 1024;
        StringBuilder sb = new StringBuilder(msgSize);
        for (int i = 0; i < msgSize; i++) {
            sb.append('a');
        }
        return sb.toString();
    }
}

堆栈跟踪。在我的函数中，第29行对应于"assertNotNull(storage.get(key));“行。

java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Arrays.java:2367)
    at java.lang.String.<init>(String.java:168)
    at org.mapdb.SerializerBase.deserializeString(SerializerBase.java:724)
    at org.mapdb.SerializerBase.deserialize(SerializerBase.java:932)
    at org.mapdb.SerializerBase.deserialize(SerializerBase.java:731)
    at org.mapdb.HTreeMap$1.deserialize(HTreeMap.java:134)
    at org.mapdb.HTreeMap$1.deserialize(HTreeMap.java:123)
    at org.mapdb.StorageDirect.recordGet2(StorageDirect.java:536)
    at org.mapdb.StorageDirect.get(StorageDirect.java:201)
    at org.mapdb.EngineWrapper.get(EngineWrapper.java:50)
    at org.mapdb.AsyncWriteEngine.get(AsyncWriteEngine.java:163)
    at org.mapdb.EngineWrapper.get(EngineWrapper.java:50)
    at org.mapdb.CacheHashTable.get(CacheHashTable.java:85)
    at org.mapdb.HTreeMap.get(HTreeMap.java:387)
    at com.sample.StorageTest.testLarge(StorageTest.java:29)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:69)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:48)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:292)

根据unholysampler的建议，我做了以下修改，解决了这个问题。

diff -r 35918e46551a src/test/java/com/sample/StorageTest.java
--- a/src/test/java/com/sample/StorageTest.java Thu Mar 21 13:40:16 2013 -0600
+++ b/src/test/java/com/sample/StorageTest.java Thu Mar 21 13:42:24 2013 -0600
@@ -16,7 +16,9 @@

     @Test
     public void testLarge() throws Exception {
-        final HTreeMap<UUID, String> storage = DBMaker.newTempHashMap();
+        File tmpFile = File.createTempFile("largeTest", null);
+        DB db = DBMaker.newFileDB(tmpFile).deleteFilesAfterClose().journalDisable().make();
+        final HTreeMap<UUID, String> storage = db.getHashMap("name");

         String string = createDataSize(250);

@@ -25,6 +27,9 @@
             final UUID key = UUID.randomUUID();
             storage.put(key, string);
             keys.add(key);
+            if (i%100==0) {
+                db.commit();
+            }
         }

         for (UUID key : keys) {

Answer 1

文档显示有一个commit()方法。

db.commit();  //persist changes into disk

testLarge()从不调用commit()，所以还没有持久化任何东西。

您也没有调用close()，因此当您转到另一个测试时，映射可能仍在内存中。

注意:我从来没有使用过这个库，但这是我最好的猜测。