cocoon-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sylvain Wallez <sylv...@apache.org>
Subject Re: Reuse of pipelines in java
Date Tue, 16 Aug 2011 20:45:14 GMT
Le 16/08/11 09:25, Steven Dolg a écrit :
> Am 14.08.2011 14:18, schrieb Sylvain Wallez:
>> Le 12/08/11 21:08, Thorsten Scherler a écrit :
>>> Hi all,
>>>
>>> I am migrating a StAX development from a customer to c3 StAX, since the
>>> resulting code will be much more generic and understandable.
>>>
>>> In my case I need to process all files from different folders, parse
>>> them and invoke a second pipeline from the main pipe.
>>>
>>> Meaning I have one principal pipeline which I need to repeat x times.
>>> I started to create the pipeline and it works very nice, however I
>>> encounter some downsides with reusing the pipe.
>>>
>>> I found that you can execute a java based pipe exactly one time. There
>>> is no such method to reset the pipe. My plan was to inject the pipeline
>>> in my main code and then configure it on the Fly (reusing the same pipe
>>> on different files).
>>>
>>> Further there is as well no way to dynamically change the different
>>> components once added to the pipe.
>>>
>>> I mean
>>>
>>> Pipeline<StAXPipelineComponent>  pipeStAX = new
>>> NonCachingPipeline<StAXPipelineComponent>();
>>> pipeStAX.addComponent(new XMLGenerator(input));
>>> ...
>>> pipeStAX.setup(System.out);
>>> pipeStAX.execute();
>>>
>>> Now my question is how people feel about:
>>> a) Making java based pipes resettable pipeStAX.reset()
>>> b) Adding a method like pipeStAX.getComonponet(int i) to retrieve the
>>> component x in position i.
>>
>
> a) What exactly should Pipeline.reset() do? (Besides calling reset on 
> each component)
> And what should a component do during a reset?
> I think components can be configured/set up as often as you like.
>
> b) If you construct the components directly, can't you keep a 
> reference to them and just call the setters/methods directly when needed?
> I guess I don't understand why the pipeline is not reusable in your 
> case or what you need to reconfigure between the runs.
> Maybe you need x different pipelines for x different configurations?
>
>
>> Although reset() can allow pipeline reuse, it won't solve the problem 
>> when you have multiple concurrent threads that could benefit from 
>> reusing the pipeline.
>>
>> Cocoon 2.x had component pools to allow reuse in a multithreaded 
>> context while avoiding the big cost of reparsing the component's 
>> configuration, but this proved to have a significant overhead.
>>
>> A solution that wouldn't require much changes in the current API 
>> would be to require pipelines and pipeline components to be 
>> Cloneable, so that you could build a pipeline instance once at 
>> startup and then clone it each time you need to use it. That would 
>> require component writers to be careful about cloneability though.
>>
>> Sylvain
>>
>
> Pipelines are not thread-safe!

Please read: I suggested to clone them, so that there is no need for 
them nor their components to be thread safe. You create a "master" 
pipeline that is never executed and then deep-clone it every time it 
needs to be processed.

Sylvain

-- 
Sylvain Wallez - http://bluxte.net


Mime
View raw message