poi-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominik Stadler <dominik.stad...@gmx.at>
Subject Re: POI memory usage degradation
Date Thu, 22 Aug 2019 16:00:16 GMT
Stacktrace indicates that this is not the latest version from trunk, are
you sure you updated?

D.

On Thu, Aug 22, 2019 at 1:16 PM Kirill <kirill.rt@gmail.com> wrote:

> Hi,
>
> Unfortunately the same OutOfMemoryError is thrown with the same stacktrace
> if I try to decrease memory allocation.
>
> Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
> at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
> at
> java.base/java.io
> .ByteArrayOutputStream.grow(ByteArrayOutputStream.java:120)
> at
> java.base/java.io
> .ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:95)
> at
> java.base/java.io
> .ByteArrayOutputStream.write(ByteArrayOutputStream.java:156)
> at
>
> org.apache.poi.openxml4j.opc.internal.MemoryPackagePartOutputStream.write(MemoryPackagePartOutputStream.java:88)
> at org.apache.xmlbeans.impl.store.Cursor._save(Cursor.java:590)
> at org.apache.xmlbeans.impl.store.Cursor.save(Cursor.java:2544)
> at
> org.apache.xmlbeans.impl.values.XmlObjectBase.save(XmlObjectBase.java:223)
> at org.apache.poi.xssf.usermodel.XSSFSheet.write(XSSFSheet.java:3651)
> at org.apache.poi.xssf.usermodel.XSSFSheet.commit(XSSFSheet.java:3572)
> at
> org.apache.poi.ooxml.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:463)
> at
> org.apache.poi.ooxml.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:468)
> at org.apache.poi.ooxml.POIXMLDocument.write(POIXMLDocument.java:236)
> at
> org.apache.poi.xssf.streaming.SXSSFWorkbook.write(SXSSFWorkbook.java:943)
>
> Best regards,
> Kirill
>
>
> On Tue, Aug 13, 2019 at 4:17 PM Dominik Stadler <dominik.stadler@gmx.at>
> wrote:
>
> > Hi,
> >
> > this should be fixed now again, see
> > https://bz.apache.org/bugzilla/show_bug.cgi?id=63657, please test it
> with
> > pre-builds from
> >
> https://builds.apache.org/view/P/view/POI/job/POI-DSL-1.8/lastSuccessfulBuild/artifact/build/dist/
> > so we are sure it fixes your use-case properly.
> >
> > Dominik.
> >
> > On Mon, Aug 5, 2019 at 10:13 AM Kirill <kirill.rt@gmail.com> wrote:
> >
> >> Dear Domenik,
> >>
> >> Could you please create the corresponding ticket for this?
> >>
> >> Best regards,
> >> Kirill
> >>
> >>
> >> On Sun, Aug 4, 2019 at 8:07 PM Dominik Stadler <dominik.stadler@gmx.at>
> >> wrote:
> >>
> >>> Hi,
> >>>
> >>> after some "git bisecting" this problem, it turns out that likely
> change
> >>> https://svn.apache.org/viewvc?view=revision&revision=1825277 is
> somehow
> >>> increasing memory usage when writing the document. It is not caused by
> >>> XMLBeans per se, but how it is used at this point and how XMLBeans
> >>> handles
> >>> arrays.
> >>>
> >>> We need to look if we can do this differently, naturally this change
> >>> tried
> >>> to fix another bug, which we don't want to re-introduce again here as
> >>> well.
> >>>
> >>> Regards... Dominik.
> >>>
> >>> On Tue, Jul 16, 2019 at 1:36 PM Kirill R <kirill.rt@gmail.com> wrote:
> >>>
> >>> > Dear POI Users Group,
> >>> >
> >>> >
> >>> > I created a smaller example regarding this issue excel-writer.zip
> with
> >>> > readme.md inside how to run it. Additionally I made a heap memory
> dump
> >>> > after OutOfMemoryError has been thrown. I assume that the root cause
> >>> of the
> >>> > issue is in the dependency xmlbeans-3.1.0.jar which was updated as
> >>> well and
> >>> > according to the heap dump the most memory is used by
> >>> > org.apache.xmlbeans.impl.store.Xobj$AttrXobj. Here is the stack trace
> >>> of
> >>> > the error:
> >>> >
> >>> > Exception in thread "main" java.lang.OutOfMemoryError: Java heap
> space
> >>> > at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
> >>> > at
> >>> > java.base/java.io
> >>> .ByteArrayOutputStream.grow(ByteArrayOutputStream.java:120)
> >>> > at
> >>> > java.base/java.io
> >>> .ByteArrayOutputStream.ensureCapacity(ByteArrayOutputStream.java:95)
> >>> > at
> >>> > java.base/java.io
> >>> .ByteArrayOutputStream.write(ByteArrayOutputStream.java:156)
> >>> > at
> >>> >
> >>>
> org.apache.poi.openxml4j.opc.internal.MemoryPackagePartOutputStream.write(MemoryPackagePartOutputStream.java:88)
> >>> > at org.apache.xmlbeans.impl.store.Cursor._save(Cursor.java:590)
> >>> > at org.apache.xmlbeans.impl.store.Cursor.save(Cursor.java:2544)
> >>> > at
> >>> >
> >>>
> org.apache.xmlbeans.impl.values.XmlObjectBase.save(XmlObjectBase.java:223)
> >>> > at
> >>> >
> >>>
> org.apache.poi.xssf.model.SharedStringsTable.writeTo(SharedStringsTable.java:279)
> >>> > at
> >>> >
> >>>
> org.apache.poi.xssf.model.SharedStringsTable.commit(SharedStringsTable.java:286)
> >>> > at
> >>> >
> >>>
> org.apache.poi.ooxml.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:463)
> >>> > at
> >>> >
> >>>
> org.apache.poi.ooxml.POIXMLDocumentPart.onSave(POIXMLDocumentPart.java:468)
> >>> > at org.apache.poi.ooxml.POIXMLDocument.write(POIXMLDocument.java:236)
> >>> > at
> >>> >
> >>>
> org.apache.poi.xssf.streaming.SXSSFWorkbook.write(SXSSFWorkbook.java:943)
> >>> > at com.excel.writer.ExcelWriter.write(ExcelWriter.java:48)
> >>> > at com.excel.writer.ExcelWriter.main(ExcelWriter.java:74)
> >>> >
> >>> >
> >>> > For the poi-ooxml with version 3.17 and its dependency
> >>> xmlbeans-2.6.0.jar,
> >>> > memory usage was twice better and it is enough to set 512MB to run
> this
> >>> > example, whereas using the latest 4.1.0 version, it requires more
> than
> >>> > twice more memory (1.1GB ) to be executed without
> >>> >
> >>> > OutOfMemoryError. It is not critical for small files like in this new
> >>> > example, but it becomes critical if the file is much bigger and
> >>> required
> >>> > 10GB or even much more memory to be written.
> >>> >
> >>> > Best regards,
> >>> > Kirill
> >>> >
> >>> >
> >>> > -----Original Message-----
> >>> > From: Mark Murphy <jmarkmurphy@gmail.com>
> >>> > Sent: Friday, 12 July, 2019 17:12
> >>> > To: POI Users List <user@poi.apache.org>
> >>> > Subject: Re: POI memory usage degradation
> >>> >
> >>> >
> >>> >
> >>> > Maybe you could take a look and see if you can narrow down what the
> >>> issues
> >>> > are. Then maybe you could contribute a patch?
> >>> >
> >>> >
> >>> >
> >>> > On Fri, Jul 12, 2019 at 11:10 AM Mark Murphy <jmarkmurphy@gmail.com>
> >>> > wrote:
> >>> >
> >>> >
> >>> >
> >>> > > I see no issue in the Bugzilla. Without an issue there, the chances
> >>> of
> >>> >
> >>> > > it being worked on or even considered are poor at best.
> >>> >
> >>> > >
> >>> >
> >>> > > On Fri, Jul 12, 2019 at 9:29 AM Kirill wrote:
> >>> >
> >>> > >
> >>> >
> >>> > >> Dear POI Users Group,
> >>> >
> >>> > >>
> >>> >
> >>> > >> Unfortunately I did not subscribe to POI Users List and did
not
> get
> >>> >
> >>> > >> any information about the raised issue.
> >>> >
> >>> > >> Now I have a subscription.
> >>> >
> >>> > >> Could you please update me regarding the issue?
> >>> >
> >>> > >>
> >>> >
> >>> > >> Best regards,
> >>> >
> >>> > >> Kirill
> >>> >
> >>> > ---------------------------------------------------------------------
> >>> > To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
> >>> > For additional commands, e-mail: user-help@poi.apache.org
> >>>
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message