cocoon-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joose Vettenranta <jo...@iki.fi>
Subject Re: pipeline executing and actions
Date Mon, 19 Sep 2005 11:13:54 GMT

Derek Hohls kirjoitti 19.9.2005 kello 9:36:

> I will also be interested in what the gurus have to say: my
> take is that the term "the whole process is streaming" is quite
> a loose one - if you're trying to process a vast chunk of data,
> won't it still take a long time, even if the data is not cached?

Yes, but if software is cachning whole data in the pipeline before  
outputting to browser, then it might take longer.. Say..

processing data takes 1hour and outputting it takes 1hour.. When  
doing both at the same time, it takes 1 hour to process, but if we  
process data first and THEN we transfer it, it will take max 2 hours.

Like generator like this:

while (somethign) {
  document.add(something else);
}

ouputDocumentAsXML();

While loops takes a long time to go, and pipelien really starts to do  
it's stuff on outputDocumentAsXML() place, right?

This is what would be nice and fast:

while (something) {
  outputAsXML(something else);
}

But when using ojb/hibernate(?) you have to mix that and xml-creating  
to do that and what I'm looking for is to keep those separate..  
Currently I pass java.util.Collection with results to jx-generator to  
create XML, but of course with BIIIG results, that is a) slow and b)  
requires lot's of memory. If I could pass java.util.Iterator to jx- 
generator and then close database connection after it's finnished,  
then a) it would be little faster and b) wouldn't require that much  
of memory. I could always put that OJB stuff to JX-template for jx- 
generator to do, but then that wouldn't be that clean xml-template...

Of course I could solve my problems by doing:

1. Custom generator (say, patch jx-generator to do my bindding)
2. Use flowscript (tested, works)
3. Make a "post serializer action" =)


Perhaps I have to start doing some custom generator then...

- Joose

>
>
>>>> joose@iki.fi 2005/09/19 07:46:49 AM >>>
>>>>
>
> Hi,
> Jonas Lundberg kirjoitti 16.9.2005 kello 17:59:
> Maybe I did not understand your problem... but if you want to do  
> something after <map:serialize>, then you can do that in a  
> different pipeline. Like this:
>
> Use a wrapper, to call the pipeline with your primary generator.
>
>
> Well, if this works, then it doesn't do what I want =)
>
> If that works, then data is buffered in generator/serializer phase  
> and then streamed out. If I have data, say.. 1000GB, then it takes  
> a very long time before any data is coming to browser (and server  
> is running out of memory). But, if the whole process is streaming  
> type (only small amount of the data is cached at any given time),  
> then it does not matter is the data 1KB or 1TB of size, right?=)
>
>
> I want to really really stream XML from server to browser.
>
> - Joose
>
> --"Always remember that you are unique, just like everyone else!"*  
> http://iki.fi/joose/ * joose@iki.fi * +358 44 561 0270 *
>
>
>
>
>
> -- 
> This message is subject to the CSIR's copyright, terms and  
> conditions and
> e-mail legal notice. Views expressed herein do not necessarily  
> represent the
> views of the CSIR.
>
> CSIR E-mail Legal Notice
> http://mail.csir.co.za/CSIR_eMail_Legal_Notice.html
>
> CSIR Copyright, Terms and Conditions
> http://mail.csir.co.za/CSIR_Copyright.html
>
> For electronic copies of the CSIR Copyright, Terms and Conditions  
> and the CSIR
> Legal Notice send a blank message with REQUEST LEGAL in the subject  
> line to
> HelpDesk@csir.co.za.
>
>
> This message has been scanned for viruses and dangerous content by  
> MailScanner,
> and is believed to be clean.  MailScanner thanks Transtec Computers  
> for their support.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
> For additional commands, e-mail: users-help@cocoon.apache.org
>
>

--
"Always remember that you are unique, just like everyone else!"
* http://iki.fi/joose/ * joose@iki.fi * +358 44 561 0270 *


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@cocoon.apache.org
For additional commands, e-mail: users-help@cocoon.apache.org


Mime
View raw message