cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thorsten Scherler <scher...@gmail.com>
Subject Re: Reuse of pipelines in java
Date Thu, 18 Aug 2011 09:39:21 GMT
On Tue, 2011-08-16 at 22:45 +0200, Sylvain Wallez wrote:
> Le 16/08/11 09:25, Steven Dolg a écrit :
> > Am 14.08.2011 14:18, schrieb Sylvain Wallez:
> >> Le 12/08/11 21:08, Thorsten Scherler a écrit :
> >>> Hi all,
> >>>
> >>> I am migrating a StAX development from a customer to c3 StAX, since the
> >>> resulting code will be much more generic and understandable.
> >>>
> >>> In my case I need to process all files from different folders, parse
> >>> them and invoke a second pipeline from the main pipe.
> >>>
> >>> Meaning I have one principal pipeline which I need to repeat x times.
> >>> I started to create the pipeline and it works very nice, however I
> >>> encounter some downsides with reusing the pipe.
> >>>
> >>> I found that you can execute a java based pipe exactly one time. There
> >>> is no such method to reset the pipe. My plan was to inject the pipeline
> >>> in my main code and then configure it on the Fly (reusing the same pipe
> >>> on different files).
> >>>
> >>> Further there is as well no way to dynamically change the different
> >>> components once added to the pipe.
> >>>
> >>> I mean
> >>>
> >>> Pipeline<StAXPipelineComponent>  pipeStAX = new
> >>> NonCachingPipeline<StAXPipelineComponent>();
> >>> pipeStAX.addComponent(new XMLGenerator(input));
> >>> ...
> >>> pipeStAX.setup(System.out);
> >>> pipeStAX.execute();
> >>>
> >>> Now my question is how people feel about:
> >>> a) Making java based pipes resettable pipeStAX.reset()
> >>> b) Adding a method like pipeStAX.getComonponet(int i) to retrieve the
> >>> component x in position i.
> >>
> >
> > a) What exactly should Pipeline.reset() do? (Besides calling reset on 
> > each component)
> > And what should a component do during a reset?
> > I think components can be configured/set up as often as you like.
> >
> > b) If you construct the components directly, can't you keep a 
> > reference to them and just call the setters/methods directly when needed?
> > I guess I don't understand why the pipeline is not reusable in your 
> > case or what you need to reconfigure between the runs.
> > Maybe you need x different pipelines for x different configurations?
> >
> >
> >> Although reset() can allow pipeline reuse, it won't solve the problem 
> >> when you have multiple concurrent threads that could benefit from 
> >> reusing the pipeline.
> >>
> >> Cocoon 2.x had component pools to allow reuse in a multithreaded 
> >> context while avoiding the big cost of reparsing the component's 
> >> configuration, but this proved to have a significant overhead.
> >>
> >> A solution that wouldn't require much changes in the current API 
> >> would be to require pipelines and pipeline components to be 
> >> Cloneable, so that you could build a pipeline instance once at 
> >> startup and then clone it each time you need to use it. That would 
> >> require component writers to be careful about cloneability though.
> >>
> >> Sylvain
> >>
> >
> > Pipelines are not thread-safe!
> 
> Please read: I suggested to clone them, so that there is no need for 
> them nor their components to be thread safe. You create a "master" 
> pipeline that is never executed and then deep-clone it every time it 
> needs to be processed.

Actually that sounds very interesting for NonCaching pipes, however the
main problem ATM for me is that I cannot execute a pipeline two times.

If that is a limitation of the pipes or a bug I do not know, but reading
Steven answer makes me think it is a bug.

salu2

-- 
Thorsten Scherler <thorsten.at.apache.org>
codeBusters S.L. - web based systems
<consulting, training and solutions>
http://www.codebusters.es/


Mime
View raw message