camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Wolf <cwolf.a...@gmail.com>
Subject Re: Why doesn't SFTP "move" option work??
Date Wed, 17 Apr 2013 21:49:59 GMT
I took the FromFtpMoveFileTest and modified it for SFTP - that works,
so "move" works
for SFTP.  My problem is that because the current release requires
hard-coding the
URI for (s)ftp(s) consumer - I implemented a custom processor that
instantiates the SFTP component,
SFTP endpoint and SFTP consumer, all programatically.  Obviously, I'm
missing something.
Unfortunately, it's pretty much a show-stopper for us to not be able
to SFTP *from* (consume)
dynamically (runtime-calculated) URIs.

On Wed, Apr 17, 2013 at 4:54 PM, Chris Wolf <cwolf.algo@gmail.com> wrote:
> I notice there's a unit test, FromFtpMoveFileTest, for plain FTP, but
> not for SFTP - should I file a bug, or am I doing something
> wrong?
>
> On Wed, Apr 17, 2013 at 11:39 AM, Chris Wolf <cwolf.algo@gmail.com> wrote:
>> Trying to use Sftp component.   I am using localWorkDirectory because
>> I don't want the whole files loaded into memory.
>> Seems like the files *are* being downloaded but they are NOT being
>> moved, on the remote side to the directory indicated by the "move"
>> option.
>>
>>
>> Here, on a poll, it gets a file, while downloading to
>> "localWorkDirectory", the file's name is appended with ".inprogress",
>> then it's
>> renamed to the base file name without ".inprogress" - so far, so good,
>> except on the server-side, it's not moved to the "move" directory.
>>
>> [#4 - sftp://localhost/download] SftpOperations DEBUG Retrieve file to
>> local work file result: true
>> [#4 - sftp://localhost/download] SftpOperations TRACE Renaming local
>> in progress file from:
>> \tmp\local\ftpwork\ADPT5_MKT_20120319_VOIR_DAILY.csv.inprogress to:
>> \tmp\local\ftpwork\ADPT5_MKT_20120319_VOIR_DAILY.csv
>> [#4 - sftp://localhost/download] SftpConsumer  TRACE Retrieved file:
>> download/ADPT5_MKT_20120319_VOIR_DAILY.csv from:
>> Endpoint[sftp://localhost/download]
>>
>> ...then, at poll n+1, I see:
>>
>> [#4 - sftp://localhost/download] SftpConsumer TRACE
>> SftpFile[fileName=ADPT5_MKT_20120319_VOIR_DAILY.csv,
>> longName=-rwxrwxrwx   1 adpt5    adpt5      298313 Apr 17 09:42
>> ADPT5_MKT_20120319_VOIR_DAILY.csv, dir=false]
>> [#4 - sftp://localhost/download] SftpConsumer TRACE Skipping as file
>> is already in progress: ADPT5_MKT_20120319_VOIR_DAILY.csv
>>
>> I guess that's the idempotent mechanism, but that file should have
>> been moved out of that directory.
>>
>>
>> SftpConfiguration:
>>
>> chmod = null,
>> ciphers = null,
>> compression = 0,
>> connectTimeout = 10000,
>> directory = download,
>> host = localhost,
>> knownHostsFile = null,
>> password = adpt5,
>> port = 22,
>> privateKeyFile = null,
>> privateKeyFilePassphrase = null,
>> protocol = sftp,
>> separator = UNIX,
>> serverAliveCountMax = 1,
>> serverAliveInterval = 0,
>> siteCommand = null,
>> soTimeout = 0,
>> strictHostKeyChecking = no,
>> timeout = 30000,
>> username = adpt5,
>> binary = true,
>> passiveMode = false,
>> stepwise = true,
>> throwExceptionOnConnectFailed = false
>>
>> [1) thread #0 - timer://kickoff] INFO  SftpEndpoint:
>> antFilter = null,
>> bufferSize = 131072,
>> camelContext = CamelContext(camel-1),
>> charset = null,
>> component = org.apache.camel.component.file.remote.SftpComponent@1e34f445,
>> configuration = download,
>> consumerProperties = null,
>> doneFileName = null,
>> endpointConfiguration =
>> org.apache.camel.impl.MappedEndpointConfiguration@8292f75a,
>> endpointKey = sftp://localhost/download,
>> endpointUri = sftp://localhost/download,
>> exchangePattern = InOnly,
>> exchanges = [Exchange[ADPT5_MKT_20090211_MICF.csv],
>> Exchange[ADPT5_MKT_20090211_MINOCF.csv],
>> Exchange[ADPT5_MKT_20090305_VOE.csv],
>> Exchange[ADPT5_MKT_20090305_VOIR.csv],
>> Exchange[ADPT5_MKT_20120319_DAILY.csv],
>> Exchange[ADPT5_MKT_20120319_FX.csv],
>> Exchange[ADPT5_MKT_20120319_MTX_DAILY.csv],
>> Exchange[ADPT5_MKT_20120319_VOIR_DAILY.csv],
>> Exchange[ADPT5_MKT_20121121_HISTORY_EUR_IRBank-FZ-D.csv],
>> Exchange[ADPT5_MKT_20121121_HISTORY_EUR_IRBank-FZ-M.csv],
>> Exchange[ADPT5_MKT_20121121_HISTORY_EUR_IRBank-FZ-Y.csv]],
>> exclude = null,
>> exclusiveReadLockStrategy = null,
>> fileExist = Override,
>> fileName = null,
>> fileSeparator = /,
>> filter = ms.algo.adapt.camel.cpmd.CPMDFileFilter@54aa2db,
>> genericFileProcessStrategy =
>> org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy@67b14530,
>> id = endpoint13,
>> idempotentRepository = null,
>> include = null,
>> inProgressRepository =
>> org.apache.camel.processor.idempotent.MemoryIdempotentRepository@563b100c,
>> localWorkDirectory = /tmp/local/ftpwork,
>> maxDepth = 2147483647,
>> maximumReconnectAttempts = 3,
>> maxMessagesPerPoll = 0,
>> minDepth = 0,
>> move = ${file:parent}/../archive/${file:onlyname},,
>> moveExisting = null,
>> moveFailed = null,
>> preMove = null,
>> processStrategy =
>> org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy@67b14530,
>> readLock = none,
>> readLockCheckInterval = 5000,
>> readLockMinLength = 1,
>> readLockTimeout = 20000,
>> reconnectDelay = 1000,
>> scheme = sftp,
>> sortBy = null,
>> sorter = null,
>> status = Started,
>> tempFileName = null,
>> tempPrefix = null,
>> version = 2.10.4,
>> allowNullBody = false,
>> autoCreate = true,
>> delete = false,
>> directoryMustExist = false,
>> disconnect = false,
>> eagerDeleteTargetFile = true,
>> eagerMaxMessagesPerPoll = true,
>> fastExistsCheck = false,
>> flatten = false,
>> keepLastModified = false,
>> lenientProperties = false,
>> noop = false,
>> recursive = false,
>> runAllowed = true,
>> singleton = true,
>> started = true,
>> starting = false,
>> startingDirectoryMustExist = false,
>> stopped = false,
>> stopping = false,
>> suspended = false,
>> suspending = false,
>> synchronous = false

Mime
View raw message