Hmm. How come when polling the absolute path it says this before listing the files in the directory
(which works)?
>> SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: /
>> SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp
>> SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: activity_logs
...but then when it tries to retrieve the file (which fails) it just says:
>> SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp
>> SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: activity_logs
That looks awfully like Camel isn't even asking the SFTP library to go to / first when it
comes time to retrieve the file.
Cheers!
-Lorrin
On Oct 5, 2010, at 12:02 AM, Claus Ibsen wrote:
> Hi
>
> Great that the relative works. This is also what I expected.
>
> In terms of absolute then Camel changes the directory to / (since its
> absolute) and the SFTP library returns OK for this.
> But in reality it does not go to the root path, but most likely just
> goes to the home directory of the user.
> And hence why absolute does not work for you.
>
> I don't see any API or way to tell SFTP library it should change to
> root path and not home folder.
>
> A solution to be able to poll files from an absolute folder is to
> create a symlink in the user home directory which can be polled by the
> SFTP library.
>
>> nlike the polling stage where it says "doPollDirectory from absolutePath: /tmp/activity_logs".
> The reason why you see this is because Camel logs the path configured
> from the endpoint, when the poller starts. Hence you see the leading
> /.
>
>
>
> On Tue, Oct 5, 2010 at 12:58 AM, Lorrin Nelson
> <lhn_github@nerdylorrin.net> wrote:
>> Hi Claus, thanks for the continued support. With the latest code (revision 1003927)
relative URIs are working again for me.
>>
>> This the first time I'm using camel-ftp. The server I'm talking to is a Fedora 11
box running OpenSSH_5.2p1. Regular FTP is an unlikely option for us. Relative URLs are an
acceptable work-around.
>>
>> Below is a trace log of an absolute URL.
>>
>> On the first attempt, note that ~/tmp does exist. Therefore when it tries to go to
*relative* path tmp (which is the wrong thing to do), it succeeds. Then when it tries to go
one level deeper into pitch_activity_logs it fails. The second attempt is really interesting.
Note there that the current working directory has remained at /home/tomcat/tmp. So when it
fails to go to *relative* path tmp, it fails right away. BTW, note that it says "Retrieving
file: tmp/activity_logs/activity_log.1285909200" with no leading slash, unlike the polling
stage where it says "doPollDirectory from absolutePath: /tmp/activity_logs".
>>
>> Cheers
>> -Lorrin
>>
>>
>> SftpConsumer 2010-10-04 15:45:07,445 -- INFO -- Connected and logged in to: sftp://tomcat@host:22
>> SftpConsumer 2010-10-04 15:45:07,494 -- TRACE -- doPollDirectory from absolutePath:
/tmp/activity_logs, dirName: null
>> SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: /
>> SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp
>> SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: activity_logs
>> SftpConsumer 2010-10-04 15:45:07,788 -- TRACE -- Polling directory: /tmp/activity_logs
>> SftpConsumer 2010-10-04 15:45:08,011 -- TRACE -- Found 4 in directory: /tmp/activity_logs
>> SftpOperations 2010-10-04 15:45:08,012 -- TRACE -- Changing directory: /
>> SftpOperations 2010-10-04 15:45:08,099 -- TRACE -- Changing directory: home
>> SftpOperations 2010-10-04 15:45:08,189 -- TRACE -- Changing directory: tomcat
>> SftpConsumer 2010-10-04 15:45:08,274 -- DEBUG -- Took 0.829 seconds to poll: /tmp/activity_logs/
>> SftpConsumer 2010-10-04 15:45:08,275 -- DEBUG -- Total 1 files to consume
>> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
>> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200
from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>> SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp
>> SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: activity_logs
>> SftpConsumer 2010-10-04 15:45:08,414 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException
- Cannot change directory to: activity_logs]
>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change
directory to: activity_logs
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:637)
>> Caused by: 2: No such file
>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
>> ... 16 more
>> ScheduledPollConsumer 2010-10-04 15:45:08,426 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>
>>
>>
>> ScheduledPollConsumer 2010-10-04 15:45:21,413 -- TRACE -- Starting to poll: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>> SftpConsumer 2010-10-04 15:45:21,413 -- TRACE -- doPollDirectory from absolutePath:
/tmp/activity_logs, dirName: null
>> SftpOperations 2010-10-04 15:45:21,413 -- TRACE -- Changing directory: /
>> SftpOperations 2010-10-04 15:45:21,496 -- TRACE -- Changing directory: tmp
>> SftpOperations 2010-10-04 15:45:21,585 -- TRACE -- Changing directory: activity_logs
>> SftpConsumer 2010-10-04 15:45:21,675 -- TRACE -- Polling directory: /tmp/activity_logs
>> SftpConsumer 2010-10-04 15:45:21,907 -- TRACE -- Found 4 in directory: /tmp/activity_logs
>> SftpOperations 2010-10-04 15:45:21,907 -- TRACE -- Changing directory: /
>> SftpOperations 2010-10-04 15:45:21,997 -- TRACE -- Changing directory: home
>> SftpOperations 2010-10-04 15:45:22,084 -- TRACE -- Changing directory: tomcat
>> SftpOperations 2010-10-04 15:45:22,177 -- TRACE -- Changing directory: tmp
>> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Took 0.848 seconds to poll: /tmp/activity_logs/
>> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Total 1 files to consume
>> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
>> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200
from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>> SftpOperations 2010-10-04 15:45:22,262 -- TRACE -- Changing directory: tmp
>> SftpConsumer 2010-10-04 15:45:22,305 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException
- Cannot change directory to: tmp]
>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change
directory to: tmp
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:637)
>> Caused by: 2: No such file
>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
>> ... 16 more
>> ScheduledPollConsumer 2010-10-04 15:45:22,311 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>>
>>
>>
>>
>> On Oct 2, 2010, at 12:14 AM, Claus Ibsen wrote:
>>
>>> Hi Loririn
>>>
>>> I think you/we have to step back. We have other users in the past
>>> running with SFTP with no problems at all. Also fetching files
>>> recursively.
>>> So I wonder if we should revert back and keep the old logic!
>>>
>>> 1)
>>> Is this the first time you use camel-ftp?
>>>
>>> 2)
>>> What OS and FTP server are you using?
>>>
>>> 3)
>>> Can you use regular FTP instead?
>>>
>>> 4)
>>> Use relative urls over absolute. They where the first supported.
>>>
>>> 5)
>>> Can you enable TRACE logging on org.apache.camel.component.file and
>>> setup a simple example so it makes it easier to follow and understand
>>> what its supposed to do, and what happens in reality so we can see and
>>> understand the difference.
>>>
>>>
>>>
>>>
>>> On Fri, Oct 1, 2010 at 10:45 PM, Lorrin Nelson
>>> <lhn_github@nerdylorrin.net> wrote:
>>>> Hi Claus. Ack, sorry about that. I'm pulling through github rather than SVN
directly and then running via a Maven project pointing at my locally built Camel, so it's
sometimes confusing how up-to-date I am.
>>>>
>>>> So I think I have current code, including checkin "CAMEL-3174: Changing dir
with ftp must do one dir at a time." and I think SFTP polling is now broken for both absolute
and relative URLs. See debug walk-throughs and stack traces below.
>>>>
>>>> Absolute URI with double leading /:
>>>> sftp://user@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>>>
>>>> SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's
no leading slash on the supposedly absolute path!
>>>> //splits into chunks successfully, starts looping through chunks
>>>> doChangeDirectory("home")
>>>> ChannelSftp.cd("home")
>>>> //path=remoteAbsolutePath(path);
>>>> //yields "/home/user/home". Doh!
>>>>
>>>>
>>>> Relative URI with single leading /:
>>>> sftp://user@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>>> SftpOperations.changeCurrentDirectory("subdir")
>>>> //splits into single chunk, starts looping through chunks
>>>> doChangeDirectory("subdir")
>>>> ChannelSftp.cd("home")
>>>> //path=remoteAbsolutePath(path);
>>>> //yields "/home/user/subdir". yay!
>>>>
>>>> ..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the
"change back to current directory code"
>>>> changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, unlike
before
>>>> //splits info chunks "", "home", and "tomcat". Doh!
>>>> doChangeDirectory("")
>>>> ChannelSftp.cd("")
>>>> //path=remoteAbsolutePath(path) throws a StringIndexOutOfBoundsException
when it receives empty string.
>>>>
>>>>
>>>> Stack trace when attempting to supply absolute path:
>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
change directory to: home
>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:637)
>>>> Caused by: 2: No such file
>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>>> ... 16 more
>>>>
>>>>
>>>> Stack trace when relative URL tries to restore current working directory:
>>>> SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException
- Cannot change directory to: ]
>>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
change directory to:
>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435)
>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>> at java.lang.Thread.run(Thread.java:637)
>>>> Caused by: 4:
>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285)
>>>> at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>>> ... 16 more
>>>> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of
range: 0
>>>> at java.lang.String.charAt(String.java:686)
>>>> at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359)
>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264)
>>>> ... 17 more
>>>>
>>>> Cheers
>>>> -Lorrin
>>>>
>>>> On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote:
>>>>
>>>>> Hi
>>>>>
>>>>> You have not used the latest source code. The stacktrace points to
>>>>> empty lines etc.
>>>>> Make sure you build camel-core and camel-ftp using latest source code.
>>>>>
>>>>> Revision: 1002946
>>>>> Node Kind: directory
>>>>> Schedule: normal
>>>>> Last Changed Author: davsclaus
>>>>> Last Changed Rev: 1002821
>>>>> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010)
>>>>>
>>>>>
>>>>> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson
>>>>> <lhn_github@nerdylorrin.net> wrote:
>>>>>> I'm pretty sure I was already at revision 1002541 before. Now I definitely
am. Same Exception. I think the problem is the lack of leading /.
>>>>>>
>>>>>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult",
calls changeCurrentDirectory(path), which in turn calls channel.cd(path), which (now we're
in com.jcraft.jsch.ChannelSftp) calls path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult",
which does not exist.
>>>>>>
>>>>>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException
- Cannot change current directory to: tmp/mult]
>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException:
Cannot change current directory to: tmp/mult
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>> Caused by: 2: No such file
>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>> ... 15 more
>>>>>>
>>>>>>
>>>>>> Cheers!
>>>>>> -Lorrin
>>>>>>
>>>>>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote:
>>>>>>
>>>>>>> Hi
>>>>>>>
>>>>>>> Can you try again with latest source code from trunk. I have
changed
>>>>>>> the logic to change dir one folder at a time.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson
>>>>>>> <lhn_github@nerdylorrin.net> wrote:
>>>>>>>> Thanks for the quick response. I'm seeing a different failure
now ("Cannot change current directory to: tmp/mult" rather than "Cannot retrieve file: tmp/mult/dummy-5.log").
Is it suspicious that in either case there is no leading /?
>>>>>>>>
>>>>>>>> I tried uris
>>>>>>>> sftp://test@host//tmp/mult?include=.*\.log
>>>>>>>> and
>>>>>>>> sftp://test@host///tmp/mult?include=.*\.log
>>>>>>>>
>>>>>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by:
[org.apache.camel.component.file.GenericFileOperationFailedException - Cannot change current
directory to: tmp/mult]
>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException:
Cannot change current directory to: tmp/mult
>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>> Caused by: 2: No such file
>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>> at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>>>> at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>>>> at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>>>> ... 15 more
>>>>>>>>
>>>>>>>> On a tangentially related note, there's some file name processing
during the move step that throws a NullPointerException when the URI contains a trailing slash:
>>>>>>>>
>>>>>>>> sftp://test@host/tmp/mult/?include=.*\.log
>>>>>>>>
>>>>>>>> The problem is that GenericFile.changeFileName calls
>>>>>>>> newFileName = ObjectHelper.after(newFileName, endpointPath
+ getFileSeparator());
>>>>>>>> ...without first checking if endPointPath contains a trailing
separator.
>>>>>>>>
>>>>>>>> E.g.
>>>>>>>> newFileName: "tmp/mult/.done/dummy-2.log"
>>>>>>>> endPointPath: "tmp/mult/"
>>>>>>>>
>>>>>>>> ObjectHelper.after():
>>>>>>>> text: "tmp/mult/.done/dummy-2.log"
>>>>>>>> after: "tmp/mult//"
>>>>>>>>
>>>>>>>> text doesn't contain after (because of double //), returns
null, changeFileName() doesn't expect null, throws NPE.
>>>>>>>>
>>>>>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR
-- Caused by: [java.lang.NullPointerException - null]
>>>>>>>> java.lang.NullPointerException
>>>>>>>> at java.io.File.<init>(File.java:222)
>>>>>>>> at org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169)
>>>>>>>> at org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41)
>>>>>>>> at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82)
>>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
>>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
>>>>>>>> at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
>>>>>>>> at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55)
>>>>>>>> at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173)
>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
>>>>>>>> at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
>>>>>>>> at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
>>>>>>>> at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
>>>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
>>>>>>>> at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
>>>>>>>> at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>> at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>> at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>>
>>>>>>>>> I have committed a fix to the SFTP component. Can you
try with latest
>>>>>>>>> source code from trunk?
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lhn_github@nerdylorrin.net>
wrote:
>>>>>>>>>>
>>>>>>>>>> I just bumped into the same problem. I'm running
2.5-SNAPSHOT.
>>>>>>>>>>
>>>>>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR --
Caused by:
>>>>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException
-
>>>>>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log]
>>>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException:
Cannot
>>>>>>>>>> retrieve file: tmp/mult/dummy-5.log
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403)
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>> at
>>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>> at java.lang.Thread.run(Thread.java:637)
>>>>>>>>>> Caused by: 2: No such file
>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>>>> at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901)
>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870)
>>>>>>>>>> at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849)
>>>>>>>>>> at
>>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400)
>>>>>>>>>> ... 14 more
>>>>>>>>>>
>>>>>>>>>> --
>>>>>>>>>> View this message in context: http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html
>>>>>>>>>> Sent from the Camel - Users mailing list archive
at Nabble.com.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> Claus Ibsen
>>>>>>>>> Apache Camel Committer
>>>>>>>>>
>>>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Claus Ibsen
>>>>>>> Apache Camel Committer
>>>>>>>
>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>
>>
>
>
>
> --
> Claus Ibsen
> Apache Camel Committer
>
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
>
|