camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <james.strac...@gmail.com>
Subject Re: Asynchronous Exchange Processing
Date Thu, 23 Aug 2007 13:34:35 GMT
On 8/22/07, Hiram Chirino <hiram@hiramchirino.com> wrote:
> Hi,
>
> Most of our components currently depend on synchronous processing of
> the Exchange or bad things can happen.  For example the following does
> not work:
>
> from("file:/tmp/foo").to("seda:test");
> from("seda:test").process( myProcessor );
>
> Why? because the file component delete the file as soon as the
> exchange returns from being sent to seda:test.  What would have been
> nice is that file deletion did not occur until after the exchange is
> processed by myProcessor.  But that's occuring in an asynchronous
> thread.
>
> Here's an idea that might help solve this problem.
> Have the seda component call something like
>    exchange.getExchangeFuture().done()
> when the message is processed in it's async thread.
>
> and in the file component, have it call
>    exchange.getExchangeFuture().get();
>    // then the code that deletes the file
> or
>    exchange.getExchangeFuture().setCallback( new Callback() {
>      public void done( Exchange exch ) {
>         // then the code that deletes the file
>      }
> })

I was pondering about this with relation to this thread the other day...
http://www.nabble.com/Consuming-FTP-file-and-deleting-after-processing-tf4300515s22882.html

I definitely think we need a standard way to register
post-commit/rollback hooks. i.e. on completion of processing (either
on a commit/completed or rollback/failed) allow a
processor/consumer/producer to register some logic such as to delete a
file, flush some cache etc. Note this is mostly required for
non-transactional things. e.g. in JPA and JMS we can just use
transactions for this.

I'm kinda wondering; should we just try make things like files, FTP
and the like transactional; that is to say, we implement transaction
hooks so that we can do a file 'delete/rename' which is registered as
a transaction commit status listener? Just registering some kind of
onCommit/onRollback callbacks would do the trick though as you
suggest.

There's a second issue which is asynchronous processing; such as a
producer invoking an asynchronous processor then wanting some kind of
callback that the processing has completed. I wanted to make the easy
things really easy with Camel; so was a bit reluctant to add
asynchronous processing explicitly from the start for fear of making
the API very complex; most components afterall tend to be synchronous
(which makes transactions very easy to do too btw).

I was thinking we could add some optional API for AsyncProcessor which
is-a Processor but adds an asynchronous invocation API style; rather
like the Channel does in the ServiceMix 4 API...

// sync API
interface Processor {
	void 	process(Exchange exchange);
}

interface AsyncProcessor extends Processor {
  // async methods
  Future<Exchange> 	processAsync(Exchange exchange)
  Future<Exchange> 	processsync(Exchange exchange, AsyncHandler handler)
}

Then rather than adding a kinda done() method to the Exchange and
calling it throughout every single producer/consumer/Processor
implementation; we could just use the Future object to know when a
particular asynchronous operation has completed. i.e. keep the async
API to the side, for those rare cases folks really wanna use it -
otherwise we can all stick to the simple sync API that works easily
with transactions.

Thoughts?
-- 
James
-------
http://macstrac.blogspot.com/

Mime
View raw message