cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jörg Heinicke (JIRA) <j...@apache.org>
Subject [jira] Commented: (COCOON-2168) ResourceReader produces Java Heap Overflow when reading a huge resource
Date Thu, 06 Mar 2008 03:50:58 GMT

    [ https://issues.apache.org/jira/browse/COCOON-2168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12575550#action_12575550
] 

Jörg Heinicke commented on COCOON-2168:
---------------------------------------

I've looked into this issue and I'm against your fix. It stands completely against the idea
of buffering the whole pipeline content which is done for error handlers. It should be possible
for them to reset the OutputStream. Or asked in other words? Why using the non-flushing BufferedOutputStream
at all?

Now how to handle the issue with huge resources? Should be quite easy. From what I see from
the code (AbstractProcessingPipeline) it is possible to configure and setup/parameterize a
pipeline with "outputBufferSize". This means on both map:pipe and map:pipeline it should be
possible to set an actual buffer size. Only if none is set (and it defaults to -1) the non-flushing
BufferedOutputStream is used.

Root sitemap has this example (in map:components/map:pipes section):

     <map:pipe name="noncaching" src="org.apache.cocoon.components.pipeline.impl.NonCachingProcessingPipeline"
               pool-max="${noncaching-pipeline.pool-max}">
       <!-- parameter name="outputBufferSize" value="8192"/ -->
     </map:pipe>

The following should also work (in map:pipelines section):

    <map:pipeline>
       <map:parameter name="outputBufferSize" value="8192"/>
       ...
    </map:pipeline>

We could argue about another default value than -1 though. Something like 1024^2.

WDYT?

> ResourceReader produces Java Heap Overflow when reading a huge resource
> -----------------------------------------------------------------------
>
>                 Key: COCOON-2168
>                 URL: https://issues.apache.org/jira/browse/COCOON-2168
>             Project: Cocoon
>          Issue Type: Bug
>          Components: * Cocoon Core
>    Affects Versions: 2.2-dev (Current SVN)
>            Reporter: Felix Knecht
>         Attachments: ResourceReader.diff, test-case.tar.gz
>
>
> When reading a huge resource (i.e. 700MB file) the ResourceReader produces an overflow
due to the BufferedOutputStream which is used (and forced to be used via AbstractReader).
The BufferedOutputStream flushes only at the end (or when forced to), but overwrites the flush
method to do nothing.
> As I don't know exactly where the BufferedOutputStream is used and what kind of impacts
it will have to change it there I'm just going to fix the ResourceReader.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message