camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From almilo <>
Subject Re: Aggregator strategies (again)
Date Mon, 05 Nov 2007 15:24:48 GMT

Hi James:

James.Strachan wrote:
> BTW Alberto I've added your test case to the distro so folks can
> noodle on it and see if we can improve things some. See
> AlbertoAggregatorTest in the camel-core module.

Many thanks, the example is a little strange but fun (hope so... :O)

James.Strachan wrote:
> The tricky thing is knowing when you're at the end of the batch
> really. One simple solution would be to add some kinda predicate to
> detect batch-completion. For example  when we split messages we could
> record how many split messages there are and each messages' counter so
> that we know when we've aggregated them all together again?

I reviewed all other replies and seem too complicated for me... (I´m new to
Camel and I chose it by simplicity = patterns + POJOs :)

I was thinking more in this simple 10.000 feet high way that you pointed out
 - Because this is a "patterns" project I think that when "splitting"
messages would be nice to count or better correlate the pieces alltogether.
Probably when theres a "split" later there is a "join".

 - I interpreted that the batchprocessor is conservative and doesn´t call
AggregationStrategy until it is really needed. It is very nice to have but
could it be possible to do AggregationStrategy more like this?:

	// newExchange: exchange that fullfils the AggregationExpression and needs
to be processed
	// AggregationContext: managed space to store intermediate data
	// Returns: the resulting exchange when finished, null when not finished 
	public Exchange aggregate(Exchange newExchange, AggregationContext

 - When a Message is splitted a "sourceMessage" property could be populated.
The splitter correlates all new messages through this value. Then a default
aggregation expression could be constructed. This is obviously too simple
and something more robust is needed but seems natural to split and join and
now this needs a lot of "glue" headers or the like to be done. Parallel
processing pipelines through POJOs and SEDA seem very easy to be built with
Camel DSL and are very powerful.

 - Using the AggregationContext you can store partial results and, when
finished (you control the batch),  the new Exchange is built and returned.
Actual strategy seems also possible to be implemented as a specialization.
Through context should also possible to know when there are more messages
pending and when started the batch so you can keep a rate of returning
exchanges or wait a little bit more.

 - This context could be transient or persistent (configured by DSL or XML)
and maybe bound to a transaction to avoid crash fails (UnitOfWork??). Could
camel manage that persistence? <- sounds not good for simplicity ;O)

I feel that these suggestions have a lot of problems with the issues of
concurrency and concepts in Camel. Am I wrong?

James, if this is good maybe I´m implementing a prototype and posting it :O)

Thanks, Alberto.
View this message in context:
Sent from the Camel - Users mailing list archive at

View raw message