FileWriter内存不足

Question

我想格式化大型Map的内容(1.785.530项)并将其写入文本文件。在大约85%的条目被处理后，它变得非常慢，然后我得到了一个OutOfMemoryException。

这个错误也会发生，即使我：

• 定期在我的flush()或close()上调用FileWriter
• 使用BufferedWriter
• 在写入文件之前，将每一行写入StringBuffer

这是我的密码：

private static final TreeMap<Date, Integer> accessesPerSecondMap = new    
    TreeMap<>();

...

private static void writeOutputFile() throws IOException {
    FileWriter writer = new FileWriter(FILENAME_OUTPUT);

    writer.write("Date");
    writer.write(',');
    writer.write("Request Count");
    writer.write('\n');

    for (Date date : accessesPerSecondMap.keySet()) {

        // first and last date are not precise so do not write it in the
        // file
        if (date == accessesPerSecondMap.firstKey()
                || date == accessesPerSecondMap.lastKey()) {
            continue;
        }

        writer.write(String.valueOf(date));
        System.out.println("FileMerger wrote: " + String.valueOf(date));
        writer.write(',');
        writer.write(String.valueOf(accessesPerSecondMap.get(date)));
        writer.write('\n');

    }

    writer.flush();
    writer.close();
}

下面是抛出的异常：

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at sun.util.resources.TimeZoneNames.getContents(Unknown Source)
    at sun.util.resources.OpenListResourceBundle.loadLookup(Unknown Source)
    at sun.util.resources.OpenListResourceBundle.loadLookupTablesIfNecessary(Unknown Source)
    at sun.util.resources.OpenListResourceBundle.handleKeySet(Unknown Source)
    at java.util.ResourceBundle.containsKey(Unknown Source)
    at sun.util.locale.provider.LocaleResources.getTimeZoneNames(Unknown Source)
    at sun.util.locale.provider.TimeZoneNameProviderImpl.getDisplayNameArray(Unknown Source)
    at sun.util.locale.provider.TimeZoneNameProviderImpl.getDisplayName(Unknown Source)
    at sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter.getName(Unknown Source)
    at sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter.getObject(Unknown Source)
    at sun.util.locale.provider.TimeZoneNameUtility$TimeZoneNameGetter.getObject(Unknown Source)
    at sun.util.locale.provider.LocaleServiceProviderPool.getLocalizedObjectImpl(Unknown Source)
    at sun.util.locale.provider.LocaleServiceProviderPool.getLocalizedObject(Unknown Source)
    at sun.util.locale.provider.TimeZoneNameUtility.retrieveDisplayName(Unknown Source)
    at java.util.TimeZone.getDisplayName(Unknown Source)
    at java.util.Date.toString(Unknown Source)
    at java.lang.String.valueOf(Unknown Source)
    at FileMerger.writeOutputFile(FileMerger.java:95)
    at FileMerger.main(FileMerger.java:26)

增加堆空间对我来说是可行的，但在我看来不是一个真正令人满意的解决方案。再过几天，我将不得不写两倍大的文件，希望空间也能满足他们的需求。

我在Windows上运行Java 1.8.0_45。上面的代码是真正的代码，也是唯一运行的代码。出于测试目的，我使用以下代码填充TreeMap

int accesses = 43267;

for (int i = 0; i < 1785530; i++) {
    Date date = new Date(i);
    accessesPerSecondMap.put(date, accesses);
}

我在for-循环中添加了以下内容，以跟踪内存使用情况：

...
    int count = 100000;
    for (Date date : accessesPerSecondMap.keySet()) {

        if (count == 100000) {
            count = 0;
            printOutMemoryUsage();
        }
        count++;
...

private void printOutMemoryUsage() {
    Runtime runtime = Runtime.getRuntime();

    NumberFormat format = NumberFormat.getInstance();

    StringBuilder sb = new StringBuilder();
    long maxMemory = runtime.maxMemory();
    long allocatedMemory = runtime.totalMemory();
    long freeMemory = runtime.freeMemory();

    sb.append("free memory: " + format.format(freeMemory / 1024) + " ");
    sb.append("allocated memory: " + format.format(allocatedMemory / 1024)
            + " ");
    sb.append("max memory: " + format.format(maxMemory / 1024) + " ");
    sb.append("total free memory: "
            + format.format((freeMemory + (maxMemory - allocatedMemory)) / 1024)
            + "<br/>");
    System.out.println(sb);
}

当我没有增加堆空间时，我得到了这个输出：

free memory: 28.868 allocated memory: 156.792 max memory: 253.440 total free memory: 125.516
free memory: 89.847 allocated memory: 253.440 max memory: 253.440 total free memory: 89.847
free memory: 87.796 allocated memory: 253.440 max memory: 253.440 total free memory: 87.796
free memory: 89.758 allocated memory: 253.440 max memory: 253.440 total free memory: 89.758
free memory: 32.478 allocated memory: 253.440 max memory: 253.440 total free memory: 32.478
free memory: 35.182 allocated memory: 253.440 max memory: 253.440 total free memory: 35.182
free memory: 37.269 allocated memory: 253.440 max memory: 253.440 total free memory: 37.269
free memory: 45.165 allocated memory: 253.440 max memory: 253.440 total free memory: 45.165
free memory: 42.943 allocated memory: 253.440 max memory: 253.440 total free memory: 42.943
free memory: 32.055 allocated memory: 253.440 max memory: 253.440 total free memory: 32.055
free memory: 13.053 allocated memory: 253.440 max memory: 253.440 total free memory: 13.053
free memory: 14.281 allocated memory: 253.440 max memory: 253.440 total free memory: 14.281
free memory: 12.797 allocated memory: 253.440 max memory: 253.440 total free memory: 12.797
free memory: 1.973 allocated memory: 253.440 max memory: 253.440 total free memory: 1.973
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

随着堆空间-Xms512m -Xmx1230m的增加，我得到：

free memory: 372.630 allocated memory: 506.816 max memory: 1.217.536 total free memory: 1.083.350
free memory: 329.031 allocated memory: 506.816 max memory: 1.217.536 total free memory: 1.039.751
free memory: 273.121 allocated memory: 506.816 max memory: 1.217.536 total free memory: 983.841
free memory: 333.700 allocated memory: 506.816 max memory: 1.217.536 total free memory: 1.044.420
free memory: 276.392 allocated memory: 506.816 max memory: 1.217.536 total free memory: 987.112
free memory: 220.482 allocated memory: 506.816 max memory: 1.217.536 total free memory: 931.202
free memory: 279.896 allocated memory: 506.816 max memory: 1.217.536 total free memory: 990.616
free memory: 223.986 allocated memory: 506.816 max memory: 1.217.536 total free memory: 934.706
free memory: 284.565 allocated memory: 506.816 max memory: 1.217.536 total free memory: 995.285
free memory: 228.654 allocated memory: 506.816 max memory: 1.217.536 total free memory: 939.374
free memory: 169.949 allocated memory: 506.816 max memory: 1.217.536 total free memory: 880.669
free memory: 230.528 allocated memory: 506.816 max memory: 1.217.536 total free memory: 941.248
free memory: 174.618 allocated memory: 506.816 max memory: 1.217.536 total free memory: 885.338
free memory: 235.197 allocated memory: 506.816 max memory: 1.217.536 total free memory: 945.917
free memory: 179.287 allocated memory: 506.816 max memory: 1.217.536 total free memory: 890.007
free memory: 123.376 allocated memory: 506.816 max memory: 1.217.536 total free memory: 834.096
free memory: 183.956 allocated memory: 506.816 max memory: 1.217.536 total free memory: 894.676
free memory: 128.046 allocated memory: 506.816 max memory: 1.217.536 total free memory: 838.766
FileMerger main method finished

Answer 1

在eclipse...on中，工具栏单击Run->Run Configuration查找您正在运行的类的名称，选择它，单击Arguments tab，然后添加：

-Xms512M -Xmx1524M // This may be XS512M or Xx1524M.

到参数部分。这将使您的大型文件能够在更大的堆空间中更顺利地运行，正如评论中所提到的@AliGajani。

Answer 2

通常有两种方法可以消除这个错误。

java.lang.OutOfMemoryError: Java heap space

1. 允许JVM使用更多内存

使用-Xmx JVM参数，您可以设置堆大小。在命令行上使用java -Xms -Xmx。

1. 使用Eclipse内存分析器是eclipse中用于分析日食内存分析器堆转储的工具。它有助于查找类加载器泄漏和内存泄漏，并帮助最小化内存，consumption.you可以使用MAT来分析承载数百万个对象的堆转储，还可以帮助您提取内存泄漏的可疑信息。

Answer 3

1. 例如，试着写前10000条记录。关闭作者，然后再次创建它，并尝试追加另10000条记录等等。
2. 尝试使用StringBuilder每次循环迭代构建一个字符串，并在循环迭代中只调用一次write()。