camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <>
Subject Re: SFTP camelExclusiveReadLock race condition
Date Fri, 15 Apr 2011 07:40:57 GMT

Is the FTP server running on Windows 2003, or is it where Camel runs?

A good strategy is of course to write using a temporary name, and then
at the end do a move/rename.
Then the consumer should "just avoid" trying to poll the "temporary files".

Camel with the rename strategy is using the rename API from the FTP
library, so if the rename success then the strategy will start
consuming the file. And that's all OS dependent. Some will not allow
the rename, while others would.

It may also depend on how the other is writing the file. There may be
IO API where you can ask for an exclusive lock on the file.

You can also try the changed lock strategy from Camel. It will check
file size / timestamp. But I recall the FTP API don't return as
precise timestamp details. But it could be worth a try. For example
you may have a long "window" where you check the file size and
timestamp don't change. And only after eg 10-30 sec. stable period,
you will try to download the file.

The "changed" hasn't been ported to FTP yet. But we love contributions :)

On Fri, Apr 15, 2011 at 12:40 AM, Patrick Daly <> wrote:
> Camel 2.7, Windows 2003
> Continuing on from this issue
> We recently moved to version 2.7.
> And are polling files from a sftp server.
> We have two client that are writing files to folder which we are scanning.
> One uploads file using a temp name followed by a rename. Everything works
> fine.
> Second client uploads file to folder without a temp name, we appear to get a
> race condition on the file
> and in turn have incomplete files transfers to our 2nd route. Even after
> setting readLock=rename
> The only files that appear to pass through without problems from the second
> client is the smaller files.
> During testing, I can drop files PDF's onto the sftp server, and quickly
> open a file.
> SFTP consumer will wait until it gets a lock before processing.
> However this lock does not appear to work whenever we have a slow producer.
> We hope to get our client to change their process.
> Thanks

Claus Ibsen
CamelOne 2011:
Twitter: davsclaus
Author of Camel in Action:

View raw message