commons-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Dudgeon <tdudg...@informaticsmatters.com>
Subject Re: [PIPELINE] Questions about pipeline
Date Fri, 24 Oct 2008 10:22:31 GMT
Ken Tanaka wrote:
> The Pipeline Basics tutorial has now been incorporated into the project 
> page. Thanks to some help and cleanup from Rahul Akolkar the 
> documentation submitted was installed quickly. See
> 
> http://commons.apache.org/sandbox/pipeline/pipeline_basics.html
> 
> -Ken
> 

That documentation is really useful. Thanks!

Could I follow up one of the earlier questions in this thread on 
branching and merging.


 From those docs it looks to me like the way data was set to a branch is 
a bit strange. There appears to be a FileReaderStage class that has Java 
bean property called htmlPipelineKey:
<stage className="com.demo.pipeline.stages.FileReaderStage"
driverFactoryId="df1" htmlPipelineKey="sales2html"/>

and later in the pipeline a branch is defined that names the pipeline 
according to that name:
<pipeline key="sales2html">

This seems pretty inflexible to me. Any branches have to be hardcoded 
into the stage definition. I was expecting a situation where multiple 
stages could be the recipients of the output of any stage, and these can 
be "wired up" dynamically. e.g. something like this:


          |--stage2
          |
stage1---+--stage3
          |
          |--stage4

so that all you needed to do was to define a stage5 as one more 
downstream stage for stage 1 and it would transparently receive the data.

Is this possible, or does the branching have to be hard-coded into the 
stage definition?


Similarly for merging. To follow up the previous question, let say I had 
stageA that output some A's and stage B that output some B's (lets 
assume both A's and B's are simple numbers). Now I wanted to have a 
stageC that takes all A's and all B's and generates some output with 
the, (lets assume the output is A * B so that every combination of A * B 
is output). So this would look like this:

stageA--+
         |
         |----stageC
         |
stageB--+

Is it possble to do this, so that stageA and stageB are both writing to 
stageC, but that stageC can distinguish the 2 different streams of data?


Many thanks.

Tim






---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@commons.apache.org
For additional commands, e-mail: user-help@commons.apache.org


Mime
View raw message