camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raul Kripalani <r...@evosent.com>
Subject Re: Splitter and combining messages / exchanges
Date Sat, 01 Dec 2012 11:00:06 GMT
I would suggest the following pipeline:

- Before the splitter, read the first line and store it in a property
- Run the splitter, and for every element concatenate the above property
with the incoming line
- Discard the first Exchange, by using a filter EIP to only allow messages
whose header CamelSplitIndex is greater than 1 or 2 (can't remember if the
index starts at 0 or 1).

Does this approach help? Just to clarify, you wouldn't be using the
aggregator EIP at all.

Regards,
Raúl.

Sent from a mobile device
On 30 Nov 2012 22:33, "Oliver Geisser" <oliver.geisser@gmail.com> wrote:

> Hi Claus,
>
> thanks for your help. I've checked the Composed Message Processor EIP.
>
> Maybe I'm missing something but as far as I can see the EIP will not work
> for my use case.
>
> Let's recap my problem with the input file
>
> Hello
> A
> B
> C
> ...
>
> Because it's a very large file it will be splitted in streaming mode.
> Therfor
> every line becomes a separate Message.
>
> Using the Aggregator the "Hello" Message will be stored in the
> AggregationRepository.
> The the second message ("A") will be aggregated with "Hello" to "Hello A".
> And now the important point: the "completion predicate" is now true and the
> meesage is released from the Aggregator, e.g.
> AggregateProcessor.onCompletion will
> be called and the "Hello" Exchange is removed from the
> AggregationRepository.
>
> The Exchange needs to be released (completed) at this point because it is a
> big file and
> we need to operate in streaming mode.
>
> Now the third message ("B") comes in. Because the "Hello" Exchange was
> removed
> we can not aggregate the Exchanges to "Hello B" anymore.
>
> So the question remains: what is the best way to solve the given problem
> (see below) with Camel?
>
> One idea I had was to mimic the Aggregate Repository but without removing
> the "Hello" Exchange
> after combining it with the "A", "B", "C", etc. messages.
>
> But is this really the best solution for my problem?
>
> What I need is access to a storage location which is scoped to the
> "Splitter input Exchange"
> during processing the "splitted" Exchanges. For example: if it would be
> possible to access
> the "input" Exchange during processing the "splitted" Exchanges I would set
> a property
> on the "input" Exchange during processing the first line and reading it
> afterwards during
> processing the remaining lines.
>
> Is there any other storage location besides an Exchange property which is
> scoped to the
> Exchange and is accessible from the "splitted" Exchanges?
>
> Greetings
> Oliver
>
>
>
> 2012/11/30 Claus Ibsen <claus.ibsen@gmail.com>
>
> > Hi
> >
> > See this EIP
> > http://camel.apache.org/composed-message-processor.html
> >
> > On Thu, Nov 29, 2012 at 11:27 PM, Oliver Geisser
> > <oliver.geisser@gmail.com> wrote:
> > > Hello,
> > >
> > > I am using a splitter to handle a big txt file (streaming mode). I need
> > to
> > > combine the first line
> > > with all the other lines and store each result into a database.
> > >
> > > What is a good Camel solution for this?
> > >
> > > Example input file:
> > >
> > > Hello
> > > A
> > > B
> > > C
> > > ...
> > >
> > >
> > > This needs to be combined into:
> > >
> > > Hello A
> > > Hello B
> > > Hello C
> > > ...
> > >
> > > Every result is stored into a database.
> > > It's important to use streaming because of the size of the input file.
> > >
> > >
> > > My first idea was to use a Splitter and then store the first line into
> an
> > > Exchange property.
> > > But this does not work because the Splitter creates new Exchanges for
> > every
> > > line.
> > >
> > > So how do I combine the first line with all the other lines?
> > >
> > > Thanks
> > > Oliver
> > >
> > > --
> > > og
> >
> >
> >
> > --
> > Claus Ibsen
> > -----------------
> > Red Hat, Inc.
> > FuseSource is now part of Red Hat
> > Email: cibsen@redhat.com
> > Web: http://fusesource.com
> > Twitter: davsclaus
> > Blog: http://davsclaus.com
> > Author of Camel in Action: http://www.manning.com/ibsen
> >
>
>
>
> --
> og
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message