在lotus文档中读取大型文件附件时，是否存在避免"java.lang.OutOfMemoryError: Java堆空间“的方法？

Question

我制作了一个java代理，它生成一个xml代码，其中包含附加在lotus文档中的powerpoint文件中的每张幻灯片文本。但是，当它尝试一个66 MB的附件时，会引发错误。

下面是代理中的完整代码

import java.io.*;
import org.apache.poi.hslf.HSLFSlideShow;
import org.apache.poi.hslf.model.Slide;
import org.apache.poi.hslf.model.TextRun;
import org.apache.poi.hslf.usermodel.SlideShow;
import org.apache.poi.poifs.filesystem.POIFSFileSystem;
import org.apache.poi.xslf.usermodel.XMLSlideShow;
import org.apache.poi.xslf.usermodel.XSLFShape;
import org.apache.poi.xslf.usermodel.XSLFSlide;
import org.apache.poi.xslf.usermodel.XSLFTextParagraph;
import org.apache.poi.xslf.usermodel.XSLFTextShape;

import lotus.domino.*;

public class JavaAgent extends AgentBase {

    public void NotesMain() {
        System.out.println("Start");
        try {
            Session session = getSession();
            AgentContext agentContext = session.getAgentContext();
            DocumentCollection dc = agentContext.getUnprocessedDocuments();
            Document doc = dc.getFirstDocument();
            Database db = agentContext.getCurrentDatabase();
            Document profdoc = db.getProfileDocument("f.SomeProfileDoc", "");
            String siteurl = profdoc.getItemValueString("SomeField");
            //System.out.println(siteurl);
            //ArrayList listFilenames = new ArrayList();
            //int indexnum = 0;
            String extension = "";
            String buildXMLStr = "";
            InputStream fis = null;
            POIFSFileSystem fs = null;
            while (doc != null) {
                RichTextItem body = (RichTextItem)doc.getFirstItem("Attachments");
                if(body != null) {
                    RichTextNavigator rtnav = body.createNavigator();
                    String filename = "";
                    if (rtnav.findFirstElement(RichTextItem.RTELEM_TYPE_FILEATTACHMENT)) {
                        buildXMLStr = "<data>";
                        do {
                            EmbeddedObject att = (EmbeddedObject) rtnav.getElement();
                            filename = att.getSource();

                            int z = filename.lastIndexOf('.');
                            if (z > 0) {
                                extension = filename.substring(z+1);
                            }

                            if(extension.equalsIgnoreCase("ppt") || extension.equalsIgnoreCase("pptx")) {

                                buildXMLStr += "\n<sfile>";
                                buildXMLStr += "<sfextension>" + extension+ "</sfextension>";
                                buildXMLStr += "\n<surl>" + siteurl + "/ContentDocs/_" + doc.getUniversalID() + "/$FILE/" + filename + "</surl>";

                                try {                                   
                                    if(extension.equalsIgnoreCase("ppt")) { // If file is a 2003 powerpoint file
                                        System.out.println("File has ppt extension");
                                        fs = new POIFSFileSystem(att.getInputStream());
                                        HSLFSlideShow show = new HSLFSlideShow(fs);
                                        SlideShow ss = new SlideShow(show);
                                        Slide[] slides = ss.getSlides();
                                        for (int x = 0; x < slides.length; x++) {

                                            buildXMLStr += "\n<slide>";
                                            buildXMLStr += "\n<snum>" + (x + 1) + "</snum>";

                                            if (slides[x].getTitle() == null) {
                                                //System.out.println("Slide " + (x + 1) + " | " + "No Title");
                                                buildXMLStr += "\n<stitle>No Title</stitle>";
                                            } else {
                                                //System.out.println("Slide " + (x + 1) + " | " + slides[x].getTitle());
                                                buildXMLStr += "\n<stitle>" + slides[x].getTitle() + "</stitle>";
                                            }

                                            buildXMLStr += "\n<stexts>";

                                            TextRun[] runs = slides[x].getTextRuns();
                                            for (int i = 0; i < runs.length; i++) {
                                                TextRun run = runs[i];
                                                buildXMLStr += " " + run.getText().replaceAll("[^\\w\\s-]", " ").replaceAll("(?m)^\\s+$", "");
                                            }

                                            buildXMLStr += "\n</stexts>";
                                            buildXMLStr += "\n</slide>";
                                        }
                                        att.getInputStream().close();
                                    }
                                    if(extension.equalsIgnoreCase("pptx")) { // If file is 2007+ powerpoint file
                                        System.out.println("File has pptx extension");
                                        try {
                                            XMLSlideShow pptxshow = new XMLSlideShow(att.getInputStream());
                                            XSLFSlide[] slides = pptxshow.getSlides();
                                            for (int x = 0; x < slides.length; x++) {

                                                buildXMLStr += "\n<slide>";
                                                buildXMLStr += "\n<snum>" + (x + 1) + "</snum>";

                                                if (slides[x].getTitle() == null) {
                                                    // System.out.println("No Title");
                                                    buildXMLStr += "\n<stitle>No Title</stitle>";
                                                } else {
                                                    // System.out.println((x + 1) + slides[x].getTitle());
                                                    buildXMLStr += "\n<stitle>" + slides[x].getTitle() + "</stitle>";
                                                }

                                                buildXMLStr += "\n<stexts>";

                                                for (XSLFShape shape : slides[x]) {
                                                    if (shape instanceof XSLFTextShape) {
                                                        XSLFTextShape txShape = (XSLFTextShape) shape;
                                                        for (XSLFTextParagraph xslfParagraph : txShape.getTextParagraphs()) {
                                                            buildXMLStr += " " + xslfParagraph.getText().replaceAll("[^\\w\\s-]", " ").replaceAll("(?m)^\\s+$", "");
                                                        }
                                                    }
                                                }

                                                buildXMLStr += "\n</stexts>";
                                                buildXMLStr += "\n</slide>";

                                            }
                                            att.getInputStream().close();
                                        } catch(Exception epptx) {
                                            epptx.printStackTrace();
                                        }

                                    }

                                } catch(Exception estart) {
                                    System.out.println("IOE Exception");
                                    estart.printStackTrace();
                                }

                                //System.out.println(filename);

                                buildXMLStr += "\n</sfile>";
                                buildXMLStr += "\n</data>";
                                System.out.println(buildXMLStr);

                            }
                            //indexnum++;
                        } while (rtnav.findNextElement());
                    }
                }
                else {
                    System.out.println("There are no attachments on the current document");
                }
                System.out.println("Next Document");
                doc = dc.getNextDocument();
            }

        } catch(Exception e) {
            System.out.println("General Exception");
            e.printStackTrace();
        }
        System.out.println("Finished scanning reports");
    }
}

Answer 1

我建议两件事：

1. 使用NPOIFSFileSystem而不是POIFSFileSystem。它使用NIO，应该消耗更少的内存。
2. 不要将输出连接到字符串中。如果必须将其保存在内存中，请使用StringBuilder，否则立即将其写入输出流。

如果所有这些都失败了，那么增加分配给JVM的内存。

Answer 2

正如Richard指出的，通过将JavaMaxHeapSize设置为大于默认值(即最大64 MB，详见此处)，可以增加代理管理器可用的内存量。

(请注意，JavaMaxHeapSize不影响HTTP。若要设置HTTP任务可用的内存量，需要设置HTTPJVMMaxHeapSize)。

Answer 3

当您试图在内存中的堆空间区域中添加更多的数据时，会触发“java.lang.OutOfMemoryError:Java堆空间”错误，但是这个数据的大小比JVM在Java堆空间中所能容纳的要大。

Java应用程序只允许使用有限的内存。此限制是在应用程序启动期间指定的。为了使事情更加复杂，Java内存被分隔到两个不同的区域。这些区域被称为堆空间和permgen。

在JVM启动期间，通过指定JVM参数(如-Xmx和-XX:MaxPermSize )来设置这些区域的大小。如果没有显式设置大小，则将使用特定于平台的默认值。

在许多情况下，如果您只是增加堆空间(在java.lang.OutOfMemoryError参数中指定)，那么消除-Xmx堆空间错误的最简单方法如下

java -Xmx1024m com.yourcompany.YourClass