Mailing-List: contact dev-help@cocoon.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cocoon.apache.org
Message-ID: <47D68114.2080406@apache.org>
Date: Tue, 11 Mar 2008 13:54:44 +0100
From: Carsten Ziegeler <cziegeler@apache.org>
User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; de;
 rv:1.8.1.12) Gecko/20080213 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666
MIME-Version: 1.0
To: dev@cocoon.apache.org
Subject: Re: [#COCOON-2168] ResourceReader produces Java Heap Overflow when
 reading a huge resource - ASF JIRA
References: <47CEC44A.80601@o-ms.com> <47CEC7FC.3030209@apache.org>
 <47CF6DB4.50200@gmx.de> <47D08520.2080806@gmx.de>
 <47D6476F.4060305@apache.org> <47D67444.9050206@gmx.de>
In-Reply-To: <47D67444.9050206@gmx.de>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Joerg Heinicke wrote:
> On 11.03.2008 04:48, Carsten Ziegeler wrote:
> 
>>>> We could argue about another default value than -1 though. Something 
>>>> like 1024^2.
>>>
>>> What do others think? Shall we change the default value from "buffer 
>>> everything" (which lead to the OutOfMemoryError [1]) to something 
>>> more "secure" in the sense of avoiding a potential source of error? 
>>> Besides mentioning it on the changes page we have to set it to a 
>>> value that's unlikely to be hit with a normal web application to 
>>> change user application's behavior only in extreme cases. That's why 
>>> I suggested 1MB.
>>>
>> Hmm, not sure if we should change the default value. The idea of this 
>> default was to be sure that error handling works out of the box. If 
>> you have special cases like mentioned in the bug, it makes imho more 
>> sense to fine tune these special cases (for instance by not buffering).
> 
> But I fear hardly anybody is aware or even uses the feature.
> 
Yes, that's possible.

>> The output buffer value is one of the settings which is "optimized" 
>> for development and it should be tweaked for production usage.
> 
> It's not really development, is it? I mean even if you can not reset 
> output buffer completely, you will still get the error markup appended 
> and I would not care for development about how this looks :)
Hmm, I would never rely on the default error handling for these cases in 
production environments. If something is already written to the output 
stream, it's too late anyway. But I see your point.

> 
> Being aware of the potential change in behavior I also chose a quite 
> huge buffer of 1 MB so that hardly anybody should be affected. We can 
> also discuss about making it even bigger like 10 MB. But I consider a 
> buffer that's flushed too early once in a while better than an OOME in 
> the default setup. And people can still change it to -1 and endless 
> buffering if they really need. But at least they are aware of the 
> effects then.
Hmm, ok, we could change this in the main sitemap as a default 
configuration while leaving it in the java code untouched.
However, I still think that this is not needed, if people want to stream 
huge responses, they should think about what they are doing and 
configure everything accordingly. I totally agree that we lack 
documentation here.

Carsten

-- 
Carsten Ziegeler
cziegeler@apache.org