poi-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominik Stadler <dominik.stad...@gmx.at>
Subject Re: POI memory usage degradation
Date Tue, 13 Aug 2019 14:17:35 GMT
Hi,

this should be fixed now again, see
https://bz.apache.org/bugzilla/show_bug.cgi?id=63657, please test it with
pre-builds from
https://builds.apache.org/view/P/view/POI/job/POI-DSL-1.8/lastSuccessfulBuild/artifact/build/dist/
so we are sure it fixes your use-case properly.

Dominik.

On Mon, Aug 5, 2019 at 10:13 AM Kirill <kirill.rt@gmail.com> wrote:

> Dear Domenik,
>
> Could you please create the corresponding ticket for this?
>
> Best regards,
> Kirill
>
>
> On Sun, Aug 4, 2019 at 8:07 PM Dominik Stadler <dominik.stadler@gmx.at>
> wrote:
>
>> Hi,
>>
>> after some "git bisecting" this problem, it turns out that likely change
>> https://svn.apache.org/viewvc?view=revision&revision=1825277 is somehow
>> increasing memory usage when writing the document. It is not caused by
>> XMLBeans per se, but how it is used at this point and how XMLBeans handles
>> arrays.
>>
>> We need to look if we can do this differently, naturally this change tried
>> to fix another bug, which we don't want to re-introduce again here as
>> well.
>>
>> Regards... Dominik.
>>
>> On Tue, Jul 16, 2019 at 1:36 PM Kirill R <kirill.rt@gmail.com> wrote:
>>
>> > Dear POI Users Group,
>> >
>> >
>> > I created a smaller example regarding this issue excel-writer.zip with
>> > readme.md inside how to run it. Additionally I made a heap memory dump
>> > after OutOfMemoryError has been thrown. I assume that the root cause of
>> the
>> > issue is in the dependency xmlbeans-3.1.0.jar which was updated as well
>> and
>> > according to the heap dump the most memory is used by
>> > org.apache.xmlbeans.impl.store.Xobj$AttrXobj. Here is the stack trace of
>> > the error:
>> >
>> > Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
>> > at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
>> > at
>> > java.base/java.io
>> .ByteArrayOutputStream.grow(ByteArrayOutputStream.java:120)
>> > at
>> > java.base/java.io
>> .ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:95)
>> > at
>> > java.base/java.io
>> .ByteArrayOutputStream.write(ByteArrayOutputStream.java:156)
>> > at
>> >
>> org.apache.poi.openxml4j.opc.internal.MemoryPackagePartOutputStream.write(MemoryPackagePartOutputStream.java:88)
>> > at org.apache.xmlbeans.impl.store.Cursor._save(Cursor.java:590)
>> > at org.apache.xmlbeans.impl.store.Cursor.save(Cursor.java:2544)
>> > at
>> >
>> org.apache.xmlbeans.impl.values.XmlObjectBase.save(XmlObjectBase.java:223)
>> > at
>> >
>> org.apache.poi.xssf.model.SharedStringsTable.writeTo(SharedStringsTable.java:279)
>> > at
>> >
>> org.apache.poi.xssf.model.SharedStringsTable.commit(SharedStringsTable.java:286)
>> > at
>> >
>> org.apache.poi.ooxml.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:463)
>> > at
>> >
>> org.apache.poi.ooxml.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:468)
>> > at org.apache.poi.ooxml.POIXMLDocument.write(POIXMLDocument.java:236)
>> > at
>> >
>> org.apache.poi.xssf.streaming.SXSSFWorkbook.write(SXSSFWorkbook.java:943)
>> > at com.excel.writer.ExcelWriter.write(ExcelWriter.java:48)
>> > at com.excel.writer.ExcelWriter.main(ExcelWriter.java:74)
>> >
>> >
>> > For the poi-ooxml with version 3.17 and its dependency
>> xmlbeans-2.6.0.jar,
>> > memory usage was twice better and it is enough to set 512MB to run this
>> > example, whereas using the latest 4.1.0 version, it requires more than
>> > twice more memory (1.1GB ) to be executed without
>> >
>> > OutOfMemoryError. It is not critical for small files like in this new
>> > example, but it becomes critical if the file is much bigger and required
>> > 10GB or even much more memory to be written.
>> >
>> > Best regards,
>> > Kirill
>> >
>> >
>> > -----Original Message-----
>> > From: Mark Murphy <jmarkmurphy@gmail.com>
>> > Sent: Friday, 12 July, 2019 17:12
>> > To: POI Users List <user@poi.apache.org>
>> > Subject: Re: POI memory usage degradation
>> >
>> >
>> >
>> > Maybe you could take a look and see if you can narrow down what the
>> issues
>> > are. Then maybe you could contribute a patch?
>> >
>> >
>> >
>> > On Fri, Jul 12, 2019 at 11:10 AM Mark Murphy <jmarkmurphy@gmail.com>
>> > wrote:
>> >
>> >
>> >
>> > > I see no issue in the Bugzilla. Without an issue there, the chances of
>> >
>> > > it being worked on or even considered are poor at best.
>> >
>> > >
>> >
>> > > On Fri, Jul 12, 2019 at 9:29 AM Kirill wrote:
>> >
>> > >
>> >
>> > >> Dear POI Users Group,
>> >
>> > >>
>> >
>> > >> Unfortunately I did not subscribe to POI Users List and did not get
>> >
>> > >> any information about the raised issue.
>> >
>> > >> Now I have a subscription.
>> >
>> > >> Could you please update me regarding the issue?
>> >
>> > >>
>> >
>> > >> Best regards,
>> >
>> > >> Kirill
>> >
>> > ---------------------------------------------------------------------
>> > To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>> > For additional commands, e-mail: user-help@poi.apache.org
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message