Thank you for the fix, but it still doesn't load. I now see two
different threads, which together take up 98% CPU time. Is there
something else I'm supposed to change (besides what you wrote in the
code)? Does it work on your machine?
andy@superlinksoftware.com wrote:
> Fix inline
>
> Mike Serra wrote:
>
>> Sorry to make this so long-winded, but I've narrowed the code down to
>> a specific test, and it's so simple I'm curious to know what could be
>> going on. Can anybody tell me why this code won't work:
>>
>> import java.io.*;
>> import org.apache.poi.hssf.usermodel.HSSFWorkbook;
>>
>> public class LoadTest {
>>
>> public static void main(String[] args) {
>>
>> HSSFWorkbook workbook;
>> File inputFile = new File("big_test.xls");
>> try {
>> InputStream fileIn = new FileInputStream(inputFile);
>
>
> fileIn = new BufferedInputStream(fileIn);
>
>>
>> workbook = new HSSFWorkbook(fileIn);
>> fileIn.close();
>> } catch (FileNotFoundException ex) {
>> } catch (IOException ex) { }
>> }
>> }
>>
>> big_test.xls is a 500kb file, with no formatted text or anything
>> unusual. When I run the above code, I can follow /proc/meminfo and
>> watch my ram slowly and continually decrease. Also, the JVM takes up
>> 98% CPU time. This goes on forever. What could possibly cause it?
>>
>> KHZ (SAW) wrote:
>>
>>> Hi Mike.
>>>
>>> I hope you find it soon. In a few days I'm away some weeks for
>>> relaxing.
>>>
>>> Regards, Karl-Heinz.
>>>
>>>
>>> -----Urspr?ngliche Nachricht-----
>>> Von: Mike E. Serra [mailto:mserra@iwu.edu] Gesendet: Montag, 21.
>>> Februar 2005 18:04
>>> An: POI Users List; karl-heinz.zengerle@sawag.com
>>> Betreff: Re: AW: AW: large xls files
>>>
>>>
>>> I think what I'll try, is to get the POI source and step the
>>> debugger into the function, and see what happens. I'll let you know
>>> (or post to the devel list) if anything interesting comes up.
>>> Thanks,
>>> Mike S.
>>>
>>> KHZ (SAW) wrote:
>>>
>>>
>>>
>>>
>>>> Hi Mike.
>>>>
>>>> It sounds like a POI bug.
>>>>
>>>> But this could also be a result of software not fitting together. As
>>>> it's not running on two platforms the latter case or overload problems
>>>> in one system seem less likely.
>>>>
>>>> You could look for similar questions in the archive or the bug
>>>> list. Ad
>>>> hoc I don't remember such an error.
>>>>
>>>> Regards, Karl-Heinz.
>>>>
>>>>
>>>> -----Urspr?ngliche Nachricht-----
>>>> Von: Mike Serra [mailto:mserra@iwu.edu] Gesendet: Montag, 21.
>>>> Februar 2005 17:47
>>>> An: POI Users List
>>>> Betreff: Re: AW: large xls files
>>>>
>>>> I have found the problem line, and it is this:
>>>>
>>>> workbook = new HSSFWorkbook(fileIn);
>>>>
>>>> The debugger steps into this line, and doesn't come out of it.
>>>> "Top" reports 98% CPU usage (I'm running linux 2.6), but this also
>>>> happens on WinXP.
>>>> So, it sounds like an infinite loop, but, why would something loop
>>>> infinitely for large files,
>>>> but work fine for small ones?
>>>> I should also note that this is not on or off behavior - for a
>>>> medium sized file, the load takes a medium amount of time, say,
>>>> thirty seconds. But enlarge the file, and load time quickly grows
>>>> to infinite. It's as if the load time increases exponentially.
>>>>
>>>> KHZ (SAW) wrote:
>>>>
>>>>
>>>>
>>>>> Hi Mike.
>>>>>
>>>>> We had files with around 3 MB which could be processed.
>>>>>
>>>>> The memory exception should come quite quickly. Possibly it's an
>>>>> infinite loop (in Windows: processor usage near 100 %, mainly used by
>>>>>
>>>>
>>>>
>>> 1
>>>
>>>
>>>>> process). You've to be sure to put the -Xmx option to the right tool.
>>>>> E.g. for debugging in Eclipse it's at at different place than for
>>>>> running jsp pages using Tomcat.
>>>>>
>>>>> Regards, Karl-Heinz.
>>>>>
>>>>>
>>>>> -----Urspr?ngliche Nachricht-----
>>>>> Von: Mike Serra [mailto:mserra@iwu.edu] Gesendet: Montag, 21.
>>>>> Februar 2005 17:24
>>>>> An: POI Users List
>>>>> Betreff: Re: large xls files
>>>>>
>>>>> My thanks to you and Karl-Heinz for your JVM tuning tips, but
>>>>> unfortunately I still can't open the file. I have tried upping both
>>>>>
>>>>
>>>>
>>>> the
>>>>
>>>>
>>>>> max heap size and the initial heap. Maybe I'm just not being
>>>>> patient enough - I think the file might load eventually but I've
>>>>> given it over
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>>>> ten minutes with no luck so far. Tell me just one thing: even
>>>>> when memory usage is high, is it normal for POI to spend a long time
>>>>>
>>>>
>>>>
>>> loading
>>>
>>>
>>>>> a file? If it isn't, the problem must be with my code.
>>>>>
>>>>> andy@superlinksoftware.com wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> I have no problems with 1000+ row files. The problem is likely
>>>>>> that
>>>>>>
>>>>>
>>>>>
>>>
>>>
>>>
>>>>>> you're likely running out of heap. java -Xmx128mb (for example)
>>>>>>
>>>>>> There are plans for doing this as NIO mem-mapped but its merely a
>>>>>> matter of funding. This plus the conversion to primatives
>>>>>> representing NUMBERs would make probably an 8-fold memory
>>>>>> decrease (I'd guess).
>>>>>>
>>>>>> -andy
>>>>>>
>>>>>> Mike Serra wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Hello out there to all POI users,
>>>>>>> I am writing a Java app which runs programmable macros on xls
>>>>>>> spreadsheets (project AutoSpreadsheet on sourceforge), and I
>>>>>>> find that large Excel files, with 1000 rows or more, simply do
>>>>>>> not load.
>>>>>>>
>>>>>>
>>>>>>
>>>
>>>
>>>
>>>>>>> I understand that there are memory issues with POI, but I didn't
>>>>>>> think it was that bad. Is there any workaround for this? If
>>>>>>> performance with POI-Ruby would be significantly better, I might
>>>>>>> simply port the whole app to Ruby. Has anyone had any
>>>>>>> experience with this?
>>>>>>>
>>>>>>> --------------------------------------------------------------------
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>> -
>>>
>>>
>>>>>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>>>>>> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
>>>>>>> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>>
>>>>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>>>>> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
>>>>>> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>>>> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
>>>>> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>>>> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
>>>>> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>>> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
>>>> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
>>>>
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>>> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
>>>> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
>>> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>>> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
>>> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
>>>
>>>
>>>
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
>> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
>> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
>> .
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: poi-user-unsubscribe@jakarta.apache.org
Mailing List: http://jakarta.apache.org/site/mail2.html#poi
The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
|