commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Dudgeon <>
Subject Re: [PIPELINE] Questions about pipeline
Date Fri, 24 Oct 2008 10:22:31 GMT
Ken Tanaka wrote:
> The Pipeline Basics tutorial has now been incorporated into the project 
> page. Thanks to some help and cleanup from Rahul Akolkar the 
> documentation submitted was installed quickly. See
> -Ken

That documentation is really useful. Thanks!

Could I follow up one of the earlier questions in this thread on 
branching and merging.

 From those docs it looks to me like the way data was set to a branch is 
a bit strange. There appears to be a FileReaderStage class that has Java 
bean property called htmlPipelineKey:
<stage className="com.demo.pipeline.stages.FileReaderStage"
driverFactoryId="df1" htmlPipelineKey="sales2html"/>

and later in the pipeline a branch is defined that names the pipeline 
according to that name:
<pipeline key="sales2html">

This seems pretty inflexible to me. Any branches have to be hardcoded 
into the stage definition. I was expecting a situation where multiple 
stages could be the recipients of the output of any stage, and these can 
be "wired up" dynamically. e.g. something like this:


so that all you needed to do was to define a stage5 as one more 
downstream stage for stage 1 and it would transparently receive the data.

Is this possible, or does the branching have to be hard-coded into the 
stage definition?

Similarly for merging. To follow up the previous question, let say I had 
stageA that output some A's and stage B that output some B's (lets 
assume both A's and B's are simple numbers). Now I wanted to have a 
stageC that takes all A's and all B's and generates some output with 
the, (lets assume the output is A * B so that every combination of A * B 
is output). So this would look like this:


Is it possble to do this, so that stageA and stageB are both writing to 
stageC, but that stageC can distinguish the 2 different streams of data?

Many thanks.


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message