sun jersey文件上传OutOfMemoryError

Question

我有一个以前可以工作的客户端代码，但随着文件变得更大，我出现了内存不足错误。我尝试了FileDataBodyPart和StreamDataBodyPart，它们都失败了，并出现了相同的错误。它是包含文件和json的多个部分。任何帮助都是非常感谢的。

下面是代码

public static Import importFile(Log notify, String importId, File file) throws CheckedContentException {
    InputStream fileStream = null;
    try {
        LOG.warn("Uploading file " + file.getName() + " with import id " + importId);
        JsonObject metadata = new JsonObject();
        metadata.addProperty("fileType", "xyz");
        metadata.add("fileOptions", new JsonObject());
        //
        // FileDataBodyPart filePart = new FileDataBodyPart(file.getName(), file);
        fileStream = Files.asByteSource(file).openStream();
        StreamDataBodyPart filePart = new StreamDataBodyPart("file", fileStream, file.getName());
        FormDataMultiPart multipart = new FormDataMultiPart();
        multipart.field("metadata", metadata.toString(), MediaType.APPLICATION_JSON_TYPE).bodyPart(filePart);
        //
        ClientResponse response = getNewWebResourceForImport(NEW_IMPORT_PATH, importId).type(MediaType.MULTIPART_FORM_DATA_TYPE)
                .accept(MediaType.APPLICATION_JSON_TYPE).post(ClientResponse.class, multipart);
        //
        String result = response.getEntity(String.class);
        Import imp = new Gson().fromJson(result, Import.class);
        if (response.getStatus() != 200 || StringUtils.equals(imp.getStatus(), IMPORT_FILE_UPLOAD_ERROR)) {
            throw new CheckedContentException("Failed to Import File with status " + response.getStatus() + " : " + response);
        }
        notify.warn("Finished uploading File " + file.getName() + " with id " + importId);
        return imp;
    } catch (CheckedContentException e) {
        throw e;
    } catch (Exception e) {
        throw new CheckedContentException(e);
    } finally {
        IOUtils.closeQuietly(fileStream);
    }
}
private static Builder getNewWebResourceForImport(String path, String importId) {
    Client client = Client.create();
    client.addFilter(new LoggingFilter(System.out));
    WebResource webResource = client.resource(WS_URL).path(path).path(importId).path("files");
    return webResource.header(ACCESS_TOKEN_KEY, API_ACCESS_TOKEN).header(CLIENT_SECRET_TOKEN_KEY, CLIENT_TOEKN);
}
java.lang.OutOfMemoryError
at java.io.ByteArrayOutputStream.hugeCapacity(Unknown Source)
at java.io.ByteArrayOutputStream.grow(Unknown Source)
at java.io.ByteArrayOutputStream.ensureCapacity(Unknown Source)
at java.io.ByteArrayOutputStream.write(Unknown Source)
at com.sun.jersey.api.client.filter.LoggingFilter$LoggingOutputStream.write(LoggingFilter.java:109)
at com.sun.jersey.core.util.ReaderWriter.writeTo(ReaderWriter.java:115)
at com.sun.jersey.core.provider.AbstractMessageReaderWriterProvider.writeTo(AbstractMessageReaderWriterProvider.java:76)
at com.sun.jersey.core.impl.provider.entity.InputStreamProvider.writeTo(InputStreamProvider.java:98)
at com.sun.jersey.core.impl.provider.entity.InputStreamProvider.writeTo(InputStreamProvider.java:59)
at com.sun.jersey.multipart.impl.MultiPartWriter.writeTo(MultiPartWriter.java:220)
at com.sun.jersey.multipart.impl.MultiPartWriter.writeTo(MultiPartWriter.java:73)
at com.sun.jersey.api.client.RequestWriter.writeRequestEntity(RequestWriter.java:300)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:217)
at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)
at com.sun.jersey.api.client.filter.LoggingFilter.handle(LoggingFilter.java:217)
at com.sun.jersey.api.client.Client.handle(Client.java:652)
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
at com.sun.jersey.api.client.WebResource$Builder.post(WebResource.java:570)

Answer 1

解决方案非常简单，我需要添加ClientConfig.PROPERTY_CHUNKED_ENCODING_SIZE并设置它。

client.setChunkedEncodingSize(32 * 1024);

它不能工作，直到我删除日志文件，因为它是耗尽内存，我添加它在第一个地方调试，但忘记了它。在研究设置块的大小时，我偶然发现了这篇文章，它也有同样的问题，并以同样的方式解决了它。希望我在挣扎之前找到了它，希望这对我有帮助。http://davidbuccola.blogspot.com/2009/09/configure-jersey-chunked-encoding-when.html