cocoon-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bruno Dumon <br...@outerthought.org>
Subject Re: Cocon 2.1: Still Problems with large downloads
Date Fri, 17 Oct 2003 17:41:28 GMT
On Fri, 2003-10-17 at 18:08, Bruno Dumon wrote:
> On Fri, 2003-10-17 at 00:24, Alexander Schatten wrote:
> > Already in Cocoon 2.0x there were severe problems with large downloads. 
> > Using a statement like:
> > 
> >  
> >             <map:match pattern="downloads/**.gz">
> >                 <map:read mime-type="application/x-gzip" 
> > src="downloads/{1}.gz"/>
> >             </map:match>
> >  
> > 
> > basically worked with files below approx. 16MB. Files with sizes above, 
> > were terminated at 16MB.
> > 
> > not good.
> > 
> > I hoped, that this is corrected in 2.1 unfortunately it seems to have a 
> > similar behaviour: I tried a file with about 24MB and the resulting 
> > downlad had 0 bytes...
> > 
> > Does anyone have an idea how to solve this problem?
> 
> I heard such claims before so I wanted to see it for myself. I simply
> added a map:read to a default Cocoon 2.1 sitemap to read a ~70 MB file.
> 
> Open a browser and surf to the correct URL... my computer slows down
> completely, I don't get anything in my browser, so I just abort jetty
> before it gets out of hand.
> 
> Then I tried putting the map:read in a pipeline like this:
> 
>   <map:pipeline type="noncaching">
>     <map:parameter name="outputBufferSize" value="8192"/>
> 
>     <map:match pattern="test.xyz">
>       <map:read src="/home/bruno/tmp/sylvain.avi"/>
>     </map:match>
>   </map:pipeline>
> 
> Two important things here:
>  * I used the non-caching pipeline
>  * I disabled complete output-buffering by specifying an
> outputBufferSize parameter with a reasonable value
> 
> which saves me about 140 MB of otherwise useless wasted RAM, and now it
> works like a charm.

Oops, forgot one aspect here: the caches and buffers aren't created of
the correct size from the first time (because it is unknown on
beforehand). Rather it starts small, lets say at 10k, and when that
buffer is full, a new one of 20k is created and the first 10k is copied
into it, then when the 20k is full one of 40k is created, and so on...
after a while it will reach e.g. 40MB and create one of 80MB, copying
the first 40MB into it, and thus needing 120MB of memory temporarely to
holds those two buffers...

-- 
Bruno Dumon                             http://outerthought.org/
Outerthought - Open Source, Java & XML Competence Support Center
bruno@outerthought.org                          bruno@apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org


Mime
View raw message