cocoon-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Bloomfield <>
Subject Re: Large XML transformations in Cocoon.
Date Sat, 20 Nov 2004 16:25:58 GMT

Thanks for the tips.  I'll move forward on coding this using STX and 
post some benchmarking numbers when I finish. 


Upayavira wrote:

> Miles Elam wrote:
> Very useful piece. Would you mind if I put this on the wiki?
> Regards, Upayavira
>> As someone who has used STX, I can recommend it in this situation 
>> wholeheartedly.  STX looks very much like XSLT but uses a different 
>> namespace and doesn't have as many options for transformation.
>> Unless something drastic has changed lately in the XSLT used by 
>> Cocoon, it uses a document table model (like a DOM but tailored 
>> toward a read-only view and a transformation source).  This is 
>> necessary because XSLT allows several passes over the same source 
>> document and also allows arbitrary access to any point in the tree 
>> (although this is usually quite inefficient).  So while XSLT is the 
>> preferred method for XML transformation in general, certain 
>> circumstances like yours would point toward alternatives.
>> As far as streaming XSLT results is concerned, it's possible to 
>> configure it this way at the expense of overall processing time.  But 
>> you don't appear to have the memory for even one full transformation 
>> let alone many at the same time.  STX is your best bet in my 
>> opinion.  This always streams the output by its very nature.
>> Also, do NOT put this into a caching pipeline.  With such a large 
>> source, memory constraints will get worse before they get better.  
>> Reprocess each time (or pregenerate on intervals a la cron) to shift 
>> the weight from memory to CPU/disk in this case.
>> Of course, a final option is to write your own custom Cocoon 
>> transformer, but I would recommend the STX route as it would likely 
>> be almost as fast and a while lot more flexible and maintainable in 
>> the long run.
>> - Miles Elam
>> On Nov 19, 2004, at 7:07 AM, Tom Bloomfield wrote:
>>> The number of iterations cooresponds to the number of rows returned 
>>> from the database.  There are roughly 46,000 rows present now, so I  
>>> need at least that many rows in my display.  The XSL design enables 
>>> me to use SAX which should help.  The easiest thing would be to 
>>> limit the number of rows returned to something more reasonable like 
>>> 10,000 (or up the JVM memory :P), but this is the requirement I'm 
>>> stuck with.
>>> Help me understand this: If I apply a transformation using XSLT, 
>>> streaming the xml in, does Cocoon "stream" the results out?   IE, 
>>> does the entire transformation happen in memory and then get flushed 
>>> to the client, or does Cocoon flush the buffer to the client as xxx 
>>> bytes are filled?  I made an assumption that Cocoon does this 
>>> automatically.
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail:
>> For additional commands, e-mail:
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message