camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: GenericFileConsumer - Idempotent and InProgress repositories
Date Mon, 13 May 2013 05:53:21 GMT
Hi

Yeah we can check the in progress repo a bit more eager in case we
have a file candidate
https://issues.apache.org/jira/browse/CAMEL-6355

Though we should do the isValidFile first as it skips unwanted file
patterns etc.

On Thu, May 9, 2013 at 8:05 PM, swwyatt <steven.wyatt@sungard.com> wrote:
> The implementations of the GenericFileConsumer (FileConsumer,
> SftpConsumer,...) method, pollDirectory(), make calls to isValidFile() and
> if that returns true, calls isInProgress(). This process fails when there
> are multiple processes polling. For example when process P0 calls
> isValidFile() and the idempotent repository does not have the file entry, it
> then calls isInProgress(). But, if process P1, who is in progress for the
> file, commits to the idempotent repository and subsequently releases its
> entry in the in progress repository, then P0 may obtain an in progress
> status and process a file that is now in the idempotent repository.
>
> Should the isInProgress() call be made before the isValidFile() call? This
> may have additional overhead of checking the in progress repository, but the
> chance for duplicate processing would be removed.
>
>
>
> --
> View this message in context: http://camel.465427.n5.nabble.com/GenericFileConsumer-Idempotent-and-InProgress-repositories-tp5732237.html
> Sent from the Camel - Users mailing list archive at Nabble.com.



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
FuseSource is now part of Red Hat
Email: cibsen@redhat.com
Web: http://fusesource.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen

Mime
View raw message