cocoon-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Bloomfield <...@shopbloomfield.com>
Subject Re: Large XML transformations in Cocoon.
Date Sat, 20 Nov 2004 16:25:58 GMT
Miles,

Thanks for the tips.  I'll move forward on coding this using STX and 
post some benchmarking numbers when I finish. 

TB



Upayavira wrote:

> Miles Elam wrote:
>
> Very useful piece. Would you mind if I put this on the wiki?
>
> Regards, Upayavira
>
>> As someone who has used STX, I can recommend it in this situation 
>> wholeheartedly.  STX looks very much like XSLT but uses a different 
>> namespace and doesn't have as many options for transformation.
>>
>> Unless something drastic has changed lately in the XSLT used by 
>> Cocoon, it uses a document table model (like a DOM but tailored 
>> toward a read-only view and a transformation source).  This is 
>> necessary because XSLT allows several passes over the same source 
>> document and also allows arbitrary access to any point in the tree 
>> (although this is usually quite inefficient).  So while XSLT is the 
>> preferred method for XML transformation in general, certain 
>> circumstances like yours would point toward alternatives.
>>
>> As far as streaming XSLT results is concerned, it's possible to 
>> configure it this way at the expense of overall processing time.  But 
>> you don't appear to have the memory for even one full transformation 
>> let alone many at the same time.  STX is your best bet in my 
>> opinion.  This always streams the output by its very nature.
>>
>> Also, do NOT put this into a caching pipeline.  With such a large 
>> source, memory constraints will get worse before they get better.  
>> Reprocess each time (or pregenerate on intervals a la cron) to shift 
>> the weight from memory to CPU/disk in this case.
>>
>> Of course, a final option is to write your own custom Cocoon 
>> transformer, but I would recommend the STX route as it would likely 
>> be almost as fast and a while lot more flexible and maintainable in 
>> the long run.
>>
>> - Miles Elam
>>
>>
>> On Nov 19, 2004, at 7:07 AM, Tom Bloomfield wrote:
>>
>>> The number of iterations cooresponds to the number of rows returned 
>>> from the database.  There are roughly 46,000 rows present now, so I  
>>> need at least that many rows in my display.  The XSL design enables 
>>> me to use SAX which should help.  The easiest thing would be to 
>>> limit the number of rows returned to something more reasonable like 
>>> 10,000 (or up the JVM memory :P), but this is the requirement I'm 
>>> stuck with.
>>>
>>> Help me understand this: If I apply a transformation using XSLT, 
>>> streaming the xml in, does Cocoon "stream" the results out?   IE, 
>>> does the entire transformation happen in memory and then get flushed 
>>> to the client, or does Cocoon flush the buffer to the client as xxx 
>>> bytes are filled?  I made an assumption that Cocoon does this 
>>> automatically.
>>
>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
>> For additional commands, e-mail: users-help@cocoon.apache.org
>>
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
> For additional commands, e-mail: users-help@cocoon.apache.org
>
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org


Mime
View raw message