camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: How to change directory while using sftp component
Date Tue, 05 Oct 2010 07:02:20 GMT
Hi

Great that the relative works. This is also what I expected.

In terms of absolute then Camel changes the directory to / (since its
absolute) and the SFTP library returns OK for this.
But in reality it does not go to the root path, but most likely just
goes to the home directory of the user.
And hence why absolute does not work for you.

I don't see any API or way to tell SFTP library it should change to
root path and not home folder.

A solution to be able to poll files from an absolute folder is to
create a symlink in the user home directory which can be polled by the
SFTP library.

> nlike the polling stage where it says "doPollDirectory from absolutePath: /tmp/activity_logs".
The reason why you see this is because Camel logs the path configured
from the endpoint, when the poller starts. Hence you see the leading
/.



On Tue, Oct 5, 2010 at 12:58 AM, Lorrin Nelson
<lhn_github@nerdylorrin.net> wrote:
> Hi Claus, thanks for the continued support. With the latest code (revision 1003927) relative
URIs are working again for me.
>
> This the first time I'm using camel-ftp. The server I'm talking to is a Fedora 11 box
running OpenSSH_5.2p1. Regular FTP is an unlikely option for us. Relative URLs are an acceptable
work-around.
>
> Below is a trace log of an absolute URL.
>
> On the first attempt, note that ~/tmp does exist. Therefore when it tries to go to *relative*
path tmp (which is the wrong thing to do), it succeeds. Then when it tries to go one level
deeper into pitch_activity_logs it fails. The second attempt is really interesting. Note there
that the current working directory has remained at /home/tomcat/tmp. So when it fails to go
to *relative* path tmp, it fails right away. BTW, note that it says "Retrieving file: tmp/activity_logs/activity_log.1285909200"
with no leading slash, unlike the polling stage where it says "doPollDirectory from absolutePath:
/tmp/activity_logs".
>
> Cheers
> -Lorrin
>
>
> SftpConsumer 2010-10-04 15:45:07,445 -- INFO -- Connected and logged in to: sftp://tomcat@host:22
> SftpConsumer 2010-10-04 15:45:07,494 -- TRACE -- doPollDirectory from absolutePath: /tmp/activity_logs,
dirName: null
> SftpOperations 2010-10-04 15:45:07,494 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-04 15:45:07,608 -- TRACE -- Changing directory: tmp
> SftpOperations 2010-10-04 15:45:07,697 -- TRACE -- Changing directory: activity_logs
> SftpConsumer 2010-10-04 15:45:07,788 -- TRACE -- Polling directory: /tmp/activity_logs
> SftpConsumer 2010-10-04 15:45:08,011 -- TRACE -- Found 4 in directory: /tmp/activity_logs
> SftpOperations 2010-10-04 15:45:08,012 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-04 15:45:08,099 -- TRACE -- Changing directory: home
> SftpOperations 2010-10-04 15:45:08,189 -- TRACE -- Changing directory: tomcat
> SftpConsumer 2010-10-04 15:45:08,274 -- DEBUG -- Took 0.829 seconds to poll: /tmp/activity_logs/
> SftpConsumer 2010-10-04 15:45:08,275 -- DEBUG -- Total 1 files to consume
> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
> SftpConsumer 2010-10-04 15:45:08,276 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200
from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
> SftpOperations 2010-10-04 15:45:08,277 -- TRACE -- Changing directory: tmp
> SftpOperations 2010-10-04 15:45:08,367 -- TRACE -- Changing directory: activity_logs
> SftpConsumer 2010-10-04 15:45:08,414 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException
- Cannot change directory to: activity_logs]
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory
to: activity_logs
>        at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
>        at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
>        at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
>        at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
>        at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>        at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>        at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>        at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>        at java.lang.Thread.run(Thread.java:637)
> Caused by: 2: No such file
>        at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>        at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>        at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>        at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
>        ... 16 more
> ScheduledPollConsumer 2010-10-04 15:45:08,426 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>
>
>
> ScheduledPollConsumer 2010-10-04 15:45:21,413 -- TRACE -- Starting to poll: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
> SftpConsumer 2010-10-04 15:45:21,413 -- TRACE -- doPollDirectory from absolutePath: /tmp/activity_logs,
dirName: null
> SftpOperations 2010-10-04 15:45:21,413 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-04 15:45:21,496 -- TRACE -- Changing directory: tmp
> SftpOperations 2010-10-04 15:45:21,585 -- TRACE -- Changing directory: activity_logs
> SftpConsumer 2010-10-04 15:45:21,675 -- TRACE -- Polling directory: /tmp/activity_logs
> SftpConsumer 2010-10-04 15:45:21,907 -- TRACE -- Found 4 in directory: /tmp/activity_logs
> SftpOperations 2010-10-04 15:45:21,907 -- TRACE -- Changing directory: /
> SftpOperations 2010-10-04 15:45:21,997 -- TRACE -- Changing directory: home
> SftpOperations 2010-10-04 15:45:22,084 -- TRACE -- Changing directory: tomcat
> SftpOperations 2010-10-04 15:45:22,177 -- TRACE -- Changing directory: tmp
> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Took 0.848 seconds to poll: /tmp/activity_logs/
> SftpConsumer 2010-10-04 15:45:22,261 -- DEBUG -- Total 1 files to consume
> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Processing file: GenericFile[activity_log.1285909200]
> SftpConsumer 2010-10-04 15:45:22,261 -- TRACE -- Retrieving file: tmp/activity_logs/activity_log.1285909200
from: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
> SftpOperations 2010-10-04 15:45:22,262 -- TRACE -- Changing directory: tmp
> SftpConsumer 2010-10-04 15:45:22,305 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException
- Cannot change directory to: tmp]
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change directory
to: tmp
>        at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:383)
>        at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:368)
>        at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:451)
>        at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:430)
>        at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>        at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>        at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>        at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>        at java.lang.Thread.run(Thread.java:637)
> Caused by: 2: No such file
>        at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>        at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>        at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>        at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:381)
>        ... 16 more
> ScheduledPollConsumer 2010-10-04 15:45:22,311 -- TRACE -- Finished polling: Endpoint[sftp://tomcat@host//tmp/activity_logs/?delay=15000&idempotent=true&idempotentRepository=%23apacheLogFilenameRepositoryAcceptNewest&include=.*.log.%5B-%2F%3A0-9%5D*&initialDelay=3000&noop=true&password=******]
>
>
>
>
> On Oct 2, 2010, at 12:14 AM, Claus Ibsen wrote:
>
>> Hi Loririn
>>
>> I think you/we have to step back. We have other users in the past
>> running with SFTP with no problems at all. Also fetching files
>> recursively.
>> So I wonder if we should revert back and keep the old logic!
>>
>> 1)
>> Is this the first time you use camel-ftp?
>>
>> 2)
>> What OS and FTP server are you using?
>>
>> 3)
>> Can you use regular FTP instead?
>>
>> 4)
>> Use relative urls over absolute. They where the first supported.
>>
>> 5)
>> Can you enable TRACE logging on org.apache.camel.component.file and
>> setup a simple example so it makes it easier to follow and understand
>> what its supposed to do, and what happens in reality so we can see and
>> understand the difference.
>>
>>
>>
>>
>> On Fri, Oct 1, 2010 at 10:45 PM, Lorrin Nelson
>> <lhn_github@nerdylorrin.net> wrote:
>>> Hi Claus. Ack, sorry about that. I'm pulling through github rather than SVN directly
and then running via a Maven project pointing at my locally built Camel, so it's sometimes
confusing how up-to-date I am.
>>>
>>> So I think I have current code, including checkin "CAMEL-3174: Changing dir with
ftp must do one dir at a time." and I think SFTP polling is now broken for both absolute and
relative URLs. See debug walk-throughs and stack traces below.
>>>
>>> Absolute URI with double leading /:
>>> sftp://user@host//home/user/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>>
>>> SftpOperations.changeCurrentDirectory("home/user/subdir") //note there's no leading
slash on the supposedly absolute path!
>>>        //splits into chunks successfully, starts looping through chunks
>>> doChangeDirectory("home")
>>> ChannelSftp.cd("home")
>>>        //path=remoteAbsolutePath(path);
>>>        //yields "/home/user/home". Doh!
>>>
>>>
>>> Relative URI with single leading /:
>>> sftp://user@host/subdir?idempotent=true&idempotentRepository=#myCustomIR&include=.*.log&noop=true&password=******
>>> SftpOperations.changeCurrentDirectory("subdir")
>>>        //splits into single chunk, starts looping through chunks
>>> doChangeDirectory("subdir")
>>> ChannelSftp.cd("home")
>>>        //path=remoteAbsolutePath(path);
>>>        //yields "/home/user/subdir". yay!
>>>
>>> ..but then, back in SftpOperations.retrieveFileToStreamInBody, it hits the "change
back to current directory code"
>>> changeCurrentDirectory("/home/tomcat"); //now we have a leading slash, unlike
before
>>>        //splits info chunks "", "home", and "tomcat". Doh!
>>> doChangeDirectory("")
>>> ChannelSftp.cd("")
>>>        //path=remoteAbsolutePath(path) throws a StringIndexOutOfBoundsException
when it receives empty string.
>>>
>>>
>>> Stack trace when attempting to supply absolute path:
>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change
directory to: home
>>>        at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>>        at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>>        at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:428)
>>>        at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>>        at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>        at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>        at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>        at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>        at java.lang.Thread.run(Thread.java:637)
>>> Caused by: 2: No such file
>>>        at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>        at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>        at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>        at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>>        ... 16 more
>>>
>>>
>>> Stack trace when relative URL tries to restore current working directory:
>>> SftpConsumer 2010-10-01 13:41:58,529 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException
- Cannot change directory to: ]
>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot change
directory to:
>>>        at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:372)
>>>        at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:361)
>>>        at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:435)
>>>        at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:407)
>>>        at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>        at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>        at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>        at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>        at java.lang.Thread.run(Thread.java:637)
>>> Caused by: 4:
>>>        at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:285)
>>>        at org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:370)
>>>        ... 16 more
>>> Caused by: java.lang.StringIndexOutOfBoundsException: String index out of range:
0
>>>        at java.lang.String.charAt(String.java:686)
>>>        at com.jcraft.jsch.ChannelSftp.remoteAbsolutePath(ChannelSftp.java:2359)
>>>        at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:264)
>>>        ... 17 more
>>>
>>> Cheers
>>> -Lorrin
>>>
>>> On Sep 29, 2010, at 9:37 PM, Claus Ibsen wrote:
>>>
>>>> Hi
>>>>
>>>> You have not used the latest source code. The stacktrace points to
>>>> empty lines etc.
>>>> Make sure you build camel-core and camel-ftp using latest source code.
>>>>
>>>> Revision: 1002946
>>>> Node Kind: directory
>>>> Schedule: normal
>>>> Last Changed Author: davsclaus
>>>> Last Changed Rev: 1002821
>>>> Last Changed Date: 2010-09-29 21:31:23 +0200 (Wed, 29 Sep 2010)
>>>>
>>>>
>>>> On Wed, Sep 29, 2010 at 11:55 PM, Lorrin Nelson
>>>> <lhn_github@nerdylorrin.net> wrote:
>>>>> I'm pretty sure I was already at revision 1002541 before. Now I definitely
am. Same Exception. I think the problem is the lack of leading /.
>>>>>
>>>>> SftpOperations.retrieveFileToStreamInBody computes path as "tmp/mult",
calls changeCurrentDirectory(path), which in turn calls channel.cd(path), which (now we're
in com.jcraft.jsch.ChannelSftp) calls path=remoteAbsolutePath(path); and computes path: "/home/test/tmp/mult",
which does not exist.
>>>>>
>>>>> SftpConsumer 2010-09-29 14:51:50,487 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException
- Cannot change current directory to: tmp/mult]
>>>>> org.apache.camel.component.file.GenericFileOperationFailedException:
Cannot change current directory to: tmp/mult
>>>>>        at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>        at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>        at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>        at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>        at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>        at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>        at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>        at java.lang.Thread.run(Thread.java:637)
>>>>> Caused by: 2: No such file
>>>>>        at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>        at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>        at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>        at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>        ... 15 more
>>>>>
>>>>>
>>>>> Cheers!
>>>>> -Lorrin
>>>>>
>>>>> On Sep 29, 2010, at 12:32 PM, Claus Ibsen wrote:
>>>>>
>>>>>> Hi
>>>>>>
>>>>>> Can you try again with latest source code from trunk. I have changed
>>>>>> the logic to change dir one folder at a time.
>>>>>>
>>>>>>
>>>>>> On Wed, Sep 29, 2010 at 7:47 PM, Lorrin Nelson
>>>>>> <lhn_github@nerdylorrin.net> wrote:
>>>>>>> Thanks for the quick response. I'm seeing a different failure
now ("Cannot change current directory to: tmp/mult" rather than "Cannot retrieve file: tmp/mult/dummy-5.log").
Is it suspicious that in either case there is no leading /?
>>>>>>>
>>>>>>> I tried uris
>>>>>>> sftp://test@host//tmp/mult?include=.*\.log
>>>>>>> and
>>>>>>> sftp://test@host///tmp/mult?include=.*\.log
>>>>>>>
>>>>>>> SftpConsumer 2010-09-29 10:31:48,401 -- ERROR -- Caused by: [org.apache.camel.component.file.GenericFileOperationFailedException
- Cannot change current directory to: tmp/mult]
>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException:
Cannot change current directory to: tmp/mult
>>>>>>>        at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:352)
>>>>>>>        at org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:408)
>>>>>>>        at org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>        at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>        at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>        at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>        at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>        at java.lang.Thread.run(Thread.java:637)
>>>>>>> Caused by: 2: No such file
>>>>>>>        at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>        at com.jcraft.jsch.ChannelSftp._realpath(ChannelSftp.java:1822)
>>>>>>>        at com.jcraft.jsch.ChannelSftp.cd(ChannelSftp.java:268)
>>>>>>>        at org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:350)
>>>>>>>        ... 15 more
>>>>>>>
>>>>>>> On a tangentially related note, there's some file name processing
during the move step that throws a NullPointerException when the URI contains a trailing slash:
>>>>>>>
>>>>>>> sftp://test@host/tmp/mult/?include=.*\.log
>>>>>>>
>>>>>>> The problem is that GenericFile.changeFileName calls
>>>>>>>        newFileName = ObjectHelper.after(newFileName, endpointPath
+ getFileSeparator());
>>>>>>> ...without first checking if endPointPath contains a trailing
separator.
>>>>>>>
>>>>>>> E.g.
>>>>>>>        newFileName: "tmp/mult/.done/dummy-2.log"
>>>>>>>        endPointPath: "tmp/mult/"
>>>>>>>
>>>>>>> ObjectHelper.after():
>>>>>>>        text: "tmp/mult/.done/dummy-2.log"
>>>>>>>        after: "tmp/mult//"
>>>>>>>
>>>>>>> text doesn't contain after (because of double //), returns null,
changeFileName() doesn't expect null, throws NPE.
>>>>>>>
>>>>>>> GenericFileOnCompletion 2010-09-29 10:37:37,888 -- ERROR -- Caused
by: [java.lang.NullPointerException - null]
>>>>>>> java.lang.NullPointerException
>>>>>>>        at java.io.File.<init>(File.java:222)
>>>>>>>        at org.apache.camel.component.file.GenericFile.changeFileName(GenericFile.java:169)
>>>>>>>        at org.apache.camel.component.file.strategy.GenericFileExpressionRenamer.renameFile(GenericFileExpressionRenamer.java:41)
>>>>>>>        at org.apache.camel.component.file.strategy.GenericFileRenameProcessStrategy.commit(GenericFileRenameProcessStrategy.java:82)
>>>>>>>        at org.apache.camel.component.file.GenericFileOnCompletion.processStrategyCommit(GenericFileOnCompletion.java:121)
>>>>>>>        at org.apache.camel.component.file.GenericFileOnCompletion.onCompletion(GenericFileOnCompletion.java:83)
>>>>>>>        at org.apache.camel.component.file.GenericFileOnCompletion.onComplete(GenericFileOnCompletion.java:52)
>>>>>>>        at org.apache.camel.util.UnitOfWorkHelper.doneSynchronizations(UnitOfWorkHelper.java:55)
>>>>>>>        at org.apache.camel.impl.DefaultUnitOfWork.done(DefaultUnitOfWork.java:173)
>>>>>>>        at org.apache.camel.processor.UnitOfWorkProcessor.doneUow(UnitOfWorkProcessor.java:121)
>>>>>>>        at org.apache.camel.processor.UnitOfWorkProcessor.access$000(UnitOfWorkProcessor.java:36)
>>>>>>>        at org.apache.camel.processor.UnitOfWorkProcessor$1.done(UnitOfWorkProcessor.java:106)
>>>>>>>        at org.apache.camel.processor.Pipeline.process(Pipeline.java:130)
>>>>>>>        at org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:99)
>>>>>>>        at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:70)
>>>>>>>        at org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:98)
>>>>>>>        at org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:89)
>>>>>>>        at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:68)
>>>>>>>        at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:322)
>>>>>>>        at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>        at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>        at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>        at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>        at java.lang.Thread.run(Thread.java:637)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sep 29, 2010, at 2:46 AM, Claus Ibsen wrote:
>>>>>>>
>>>>>>>> Hi
>>>>>>>>
>>>>>>>> I have committed a fix to the SFTP component. Can you try
with latest
>>>>>>>> source code from trunk?
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Sep 28, 2010 at 7:36 PM, Lorrin <lhn_github@nerdylorrin.net>
wrote:
>>>>>>>>>
>>>>>>>>> I just bumped into the same problem. I'm running 2.5-SNAPSHOT.
>>>>>>>>>
>>>>>>>>> SftpConsumer 2010-09-28 10:31:28,094 -- ERROR -- Caused
by:
>>>>>>>>> [org.apache.camel.component.file.GenericFileOperationFailedException
-
>>>>>>>>> Cannot retrieve file: tmp/mult/dummy-5.log]
>>>>>>>>> org.apache.camel.component.file.GenericFileOperationFailedException:
Cannot
>>>>>>>>> retrieve file: tmp/mult/dummy-5.log
>>>>>>>>>        at
>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:403)
>>>>>>>>>        at
>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFile(SftpOperations.java:387)
>>>>>>>>>        at
>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:299)
>>>>>>>>>        at
>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:155)
>>>>>>>>>        at
>>>>>>>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:121)
>>>>>>>>>        at
>>>>>>>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:97)
>>>>>>>>>        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
>>>>>>>>>        at
>>>>>>>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
>>>>>>>>>        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
>>>>>>>>>        at
>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
>>>>>>>>>        at
>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:181)
>>>>>>>>>        at
>>>>>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:205)
>>>>>>>>>        at
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>>>>>>>        at
>>>>>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>>>>>>>        at java.lang.Thread.run(Thread.java:637)
>>>>>>>>> Caused by: 2: No such file
>>>>>>>>>        at com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2289)
>>>>>>>>>        at com.jcraft.jsch.ChannelSftp._get(ChannelSftp.java:901)
>>>>>>>>>        at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:870)
>>>>>>>>>        at com.jcraft.jsch.ChannelSftp.get(ChannelSftp.java:849)
>>>>>>>>>        at
>>>>>>>>> org.apache.camel.component.file.remote.SftpOperations.retrieveFileToStreamInBody(SftpOperations.java:400)
>>>>>>>>>        ... 14 more
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> View this message in context: http://camel.465427.n5.nabble.com/How-to-change-directory-while-using-sftp-component-tp2806817p2857187.html
>>>>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Claus Ibsen
>>>>>>>> Apache Camel Committer
>>>>>>>>
>>>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>>>> Open Source Integration: http://fusesource.com
>>>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Claus Ibsen
>>>>>> Apache Camel Committer
>>>>>>
>>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>>> Open Source Integration: http://fusesource.com
>>>>>> Blog: http://davsclaus.blogspot.com/
>>>>>> Twitter: http://twitter.com/davsclaus
>>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Claus Ibsen
>>>> Apache Camel Committer
>>>>
>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>> Open Source Integration: http://fusesource.com
>>>> Blog: http://davsclaus.blogspot.com/
>>>> Twitter: http://twitter.com/davsclaus
>>>>
>>>
>>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Mime
View raw message