cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carsten Ziegeler <cziege...@apache.org>
Subject Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when reading a huge resource - ASF JIRA
Date Tue, 11 Mar 2008 09:29:54 GMT
Felix Knecht schrieb:
> Carsten Ziegeler schrieb:
>> Joerg Heinicke wrote:
>>> On 05.03.2008 23:06, Joerg Heinicke wrote:
>>>
>>>> We could argue about another default value than -1 though. Something 
>>>> like 1024^2.
>>>
>>> What do others think? Shall we change the default value from "buffer 
>>> everything" (which lead to the OutOfMemoryError [1]) to something 
>>> more "secure" in the sense of avoiding a potential source of error? 
>>> Besides mentioning it on the changes page we have to set it to a 
>>> value that's unlikely to be hit with a normal web application to 
>>> change user application's behavior only in extreme cases. That's why 
>>> I suggested 1MB.
>>>
>> Hmm, not sure if we should change the default value. The idea of this 
>> default was to be sure that error handling works out of the box. If 
>> you have special cases like mentioned in the bug, it makes imho more 
>> sense to fine tune these special cases (for instance by not buffering).
>>
>> The output buffer value is one of the settings which is "optimized" 
>> for development and it should be tweaked for production usage. I think 
>> also if you're using a reader in your pipeline it is more likely that 
>> you don't want to let the pipeline buffer your output.
> 
> IIUC this should work (no caching pipeline, set buffer-size to 8192):
> 
>    <map:pipeline id="test-nocache" type="noncaching">
>      <map:match pattern="nocache">
>        <map:read src="/home/felix/tmp/livecd-i686-installer-2007.0.iso" >
>          <map:parameter name="buffer-size" value="8192" />
>        </map:read>
>      </map:match>
>    </map:pipeline>
> 
> but it doesn't
> 
> java.lang.OutOfMemoryError: Java heap space

You need to turn off the buffering of the pipeline as well. I don't have 
the parameter name at hand, I assume it's "buffer-size" as well but 
could be different:

     <map:pipeline id="test-nocache" type="noncaching">
       <map:parameter name="buffer-size" value="0" />
       <map:match pattern="nocache">
         <map:read src="/home/felix/tmp/livecd-i686-installer-2007.0.iso"
           <map:parameter name="buffer-size" value="8192" />
         </map:read>
       </map:match>
     </map:pipeline>

This is usually the way I would define reader pipelines. If the above 
still produces an OOMError than we have a bug :)

Carsten
-- 
Carsten Ziegeler
cziegeler@apache.org

Mime
View raw message