camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From camel_el <eric.ladouc...@cse-cst.gc.ca>
Subject Re: Camel calling commit too early when using split+seda+file endpoint
Date Thu, 09 Sep 2010 20:07:03 GMT

This piece of code is a very simplified version of my real code.  My
application is running on an OSGi container and the different stages in my
processing are isolated in different bundles.  So when I need to pass a
message from one stage to another, I need to use a 'vm' endpoint, a.k.a. a
seda queue.

To give you more background, I am processing large files (each file can
contain up to 500,000 items) and I receive a lot of those.  All of my input
is multithreaded, so that I can read many files at the same time.

overview of my real route
from("file:input....")
   .threads(5)
   .bean(ObjectIteratorFactory.class, "createIterator")  // this bean
creates an iterator that will read 1 item at a time
                                                                // and it
will keep the input file opened until it is done
   .split(body()).streaming()
      .process(firstProcessor)
      .to("vm:secondStageInAnotherBundle")
   .end()
   .log("Done processing file ${file:name}")

So that seda queue (or vm endpoint in my case) is needed since I don't want
to have all of my code inside the same bundle.  If I don't use a seda queue,
it works fine and the file gets deleted at then end, when ObjectIterator has
closed its input stream.  If the seda is involved, Camel tries to close the
file after processing the first item in the file...
-- 
View this message in context: http://camel.465427.n5.nabble.com/Camel-calling-commit-too-early-when-using-split-seda-file-endpoint-tp2830894p2834071.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Mime
View raw message