cocoon-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bruce Atherton <br...@callenish.com>
Subject Avoiding OutOfMemory Errors by limiting data in pipeline
Date Thu, 24 Apr 2008 00:56:23 GMT
I have been running into a real problem with OutOfMemory errors on 
Cocoon 2.1.9. The trouble seems to be that occasionally users request 
settings for some reports that end up killing the server with an 
OutOfMemory error. This also leaves us open to denial of service attacks.

My question is whether there is a way to throw an exception or otherwise 
limit the processing of these large payloads from within the pipeline?

Given the volume of data available for legitimate reporting in my 
application it is not possible to limit the request parameters in the 
application in any sensible way, and the JVMs are already running with 
Gigs of heap. Pagination is not an option for many of the reports. I 
could limit the number of objects returned if I could find all the 
reports that could potentially cause this problem, but it would be much 
simpler to just stop Cocoon from processing data that is too large.

Here are some specifics, in case they are relevant. One heap analysis 
showed 1.5 Gigabytes of memory being held by the object at the end of 
this tree (package names suppressed to keep this readable):

  - ScriptableObject
    - FOM_Cocoon
      - FOM_Cocoon$CallContext
        - ConcreteTreeProcessor
          - InvokeContext
            - CachingProcessorPipeline
              - ArrayList
                - Object
                  - ComponentsSelector
                    - ComponentsSelector
                      - Collections$SynchronizedMap
                        - HashMap
                          - Array of HashMap$Entry
                            - HashMap$Entry (size including children = 
1.5Gig)

That HashMap$Entry object had the following child tree, each one a bit 
smaller than its parent. Only the largest child shown at each level:

  - HashMap$Entry (a different one)
    - TraxTransformer
      - TransformerHandlerImpl
        - SAXResult
          - JxTemplateTransformer
            - JxTemplateGenerator$TranformerAdapter$TemplateConsumer

After this there are two JxTemplateGenerators that split the size. The 
bigger one goes to FormsTemplateTransformer, CIncludeTransformer, and on 
and on into a long loop of JxTemplateGenerator$StartElement and 
JxTemplateGenerator$EndElement.

It seems to me that it should be possible to put a limiter on the 
transformers that says that if more than a certain volume of data is put 
through, an exception is thrown. I know that part of the page may have 
been rendered before the error page shows up, but I am fine with that. 
I'd be happy to limit each part of the pipeline to outputting no more 
than 10Meg.

Can anyone make any suggestions as to how to make this work in Cocoon? 
Do I need to hack on JxTransformer? Configure Xalan to limit data size? 
Make a configuration change to Cocoon? Any help is appreciated.

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org


Mime
View raw message