poi-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yegor Kozlov <ye...@dinom.ru>
Subject Re: Any solution to eliminate out of memory error for large export to excel apart from increasing the heap size.
Date Tue, 27 Oct 2009 09:00:24 GMT
> Let's say I have 100000 records to be exported,
> Now in one sheet of a workbook it can contain only 65535 records,
> Then the other set of 34465 records should get created in another sheet
> of the same workbook.
> 
> Is it possible in Yegor's BigGridDemo? Using the xml approach?
> 
Yes, it is possible. The BigGridDemo application streams the generated data into a temporary
file and then injects it in 
the target OOXML package. This approach allows generating really large sets of data, up to
the grid limits.
Splitting the data across multiple sheets is possible, actually it requires only a few lines
of code to add.

Yegor

> Thanks,
> Santosh
> 
> -----Original Message-----
> From: MSB [mailto:markbrdsly@tiscali.co.uk] 
> Sent: Tuesday, October 27, 2009 12:58 PM
> To: user@poi.apache.org
> Subject: RE: Any solution to eliminate out of memory error for large
> export to excel apart from increasing the heap size.
> 
> 
> I do not remember saying that Yegor's BigGridDemo demonstrated the ONLY
> way
> to proceed with larger data-sets merely that it demonstrated a possible
> approach if you working with the OpenXML based file format.
> 
> Having said that, can I just be a little more clear about what you are
> asking please? Are you saying that you have sufficient data to populate
> more
> than sixty five thousand rows and that rather than see this data
> converted
> into a single Excel worksheet, you want to see it split across several
> worksheets? If so, how should the data be split or divided; what
> criteria
> should determine which sheet each row is written to?
> 
> Yours
> 
> Mark B
> 
> 
> Dronamraju, Santosh B (GE Indust, ConsInd) wrote:
>> I am using POI V3.5, and was creating binary .xls files.
>>
>> Now that I understand large export are possible only using xml based
>> .xlsx files approach, thanks for you help.
>>
>> I have another question, for an export more than 65,535 records, how
> to
>> instruct xml to open up in a separate sheet instead of not showing the
>> rest of the records?
>>
>> Thanks,
>> Santosh
>> -----Original Message-----
>> From: MSB [mailto:markbrdsly@tiscali.co.uk] 
>> Sent: Friday, October 23, 2009 7:57 PM
>> To: user@poi.apache.org
>> Subject: Re: Any solution to eliminate out of memory error for large
>> export to excel apart from increasing the heap size.
>>
>>
>> May I ask which version of the Excel file format you are targetting?
> Are
>> you
>> trying to create binary .xls files or xml based .xlsx files? If the
>> latter
>> then you could take a look at the
>>
> http://svn.apache.org/repos/asf/poi/trunk/src/examples/src/org/apache/po
>> i/xssf/usermodel/examples/BigGridDemo.java
>> proof that Yegor created. It is only intended to demonstrate a
> technique
>> and
>> must not be thought of as a complete application however.
>>
>> If, however, you are targetting the binary file format then I am
> afraid
>> that
>> the news is not too good. Owing to the file format - it consists of a
>> series
>> of linked lists - it is not possible to write the file out in chunks.
>>
>> Yours
>>
>> Mark B
>>
>>
>> Dronamraju, Santosh B (GE Indust, ConsInd) wrote:
>>> Hi everyone,
>>>
>>>  
>>>
>>> For large excel exports, I get out of memory error. Is there any way
>> to
>>> write directly to physical file record by record instead of using
> heap
>>> and then writing the whole data to file system at the end?
>>>
>>>  
>>>
>>> Because increasing heap size is not working even if I use 1024 MB. As
>> in
>>> our application we have large and number of export to excel.
>>>
>>>  
>>>
>>> Thanks,
>>>
>>> Santosh
>>>
>>>  
>>>
>>>
>>>
>> -- 
>> View this message in context:
>>
> http://www.nabble.com/Any-solution-to-eliminate-out-of-memory-error-for-
> large-export-to-excel-apart-from-increasing-the-heap-size.-tp26022706p26
>> 027366.html
>> Sent from the POI - User mailing list archive at Nabble.com.
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>> For additional commands, e-mail: user-help@poi.apache.org
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
>> For additional commands, e-mail: user-help@poi.apache.org
>>
>>
>>
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@poi.apache.org
For additional commands, e-mail: user-help@poi.apache.org


Mime
View raw message