camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Diether <diether.wu...@optis.be>
Subject RE: Problems with sftp in Camel 2.11.0
Date Wed, 24 Apr 2013 08:53:51 GMT

This problem has been mentioned before:
http://camel.465427.n5.nabble.com/Producing-files-to-ftp-now-working-in-2-10-3-td5726060.html

But it seems like still no fix has been implemented then.
For us it was working in 2.10.2 though, but not in 2.10.3

Regards,
Diether

________________________________
Van: Bengt Rodehav [via Camel] [ml-node+s465427n5731408h95@n5.nabble.com]
Verzonden: woensdag 24 april 2013 10:28
To: Wuyts Diether
Onderwerp: Re: Problems with sftp in Camel 2.11.0

I investigated this further and found a serious bug in the
FileUtil.compactPath() method. If the path only contains a "/" (or "\" on
Windows) the compactPath() method will:

- Determine that the path both starts and ends with a slash.
- First output a slash because the path starts with a slash
- Then output whatever comes after the leading slash (in this case nothing)
- Last output a slash because the path ends with a slash, thus ending up
with two slashes which is incorrect

The logic always assume that there is some other content in between the
starting and ending slash. This will of course fail if the path only
consists of a single slash (or backslash).

>From what I can see this error was introduced in revision 1389069 on
September 23, 2012. This means that it probably doesn't work on  any
version from (and including) Camel 2.10.2. The version we have in
production is Camel 2.7.1 which is why I haven't seen this problem before.
I was now hoping to upgrade to the latest Camel (2.11.0) which is when I
ran into this problem.

This seems like a major blocker to me. I suspect that most people uploading
files with sftp will encounter this problem since you normally have "/" as
your home directory. Furthermore, the FileUtil class resides in camel-core
which means that a lot of other components than sftp might be affected.

Can anyone verify this? Is it possible to have a quick patch release (if
I'm right?)

/Bengt


2013/4/24 Bengt Rodehav <[hidden email]<UrlBlockedError.aspx>>

> The following trace logging looks suspicious to me:
>
> 2013-04-24 09:34:40,776 | TRACE | terfaces/nfm/out | SftpOperations
>             | onent.file.remote.SftpOperations  377 | Current dir: /
> 2013-04-24 09:34:40,782 | TRACE | terfaces/nfm/out | SftpOperations
>             | onent.file.remote.SftpOperations  385 |
> changeCurrentDirectory(/)
> 2013-04-24 09:34:40,782 | TRACE | terfaces/nfm/out | SftpOperations
>             | onent.file.remote.SftpOperations  396 | Compacted path: / ->
> // using separator: /
> 2013-04-24 09:34:40,782 | TRACE | terfaces/nfm/out | SftpOperations
>             | onent.file.remote.SftpOperations  440 | Changing directory: //
>
> So, it seems like Camel correctly determines that current directory is "/"
> but it then changes the path to "//" which of course is incorrect.
>
> /Bengt
>
>
> 2013/4/24 Bengt Rodehav <[hidden email]<UrlBlockedError.aspx>>
>
>> Some more information. I logged on the sftp server using cygwin (I'm
>> running on Windows 7 BTW) and executed a "pwd" command just to see what the
>> server replied and it said:
>>
>> sftp> pwd
>> Remote working directory: /
>>
>> This looks perfectly fine to me and is a normal setup. I e my home
>> directory looks like the root folder to me since I'm not authorised to
>> anything but my home directory. Don't quite understand what causes my
>> problems.
>>
>> /Bengt
>>
>>
>> 2013/4/24 Bengt Rodehav <[hidden email]<UrlBlockedError.aspx>>
>>
>>> I've now checked this issue a bit further. It turned out that I hadn't
>>> tested this on Camel 2.10.3. What I had tested was the corresponding
>>> consumtion of files from the sftp server which also works with Camel
>>> 2.11.0. So this is probably not a Camel 2.11.0 issue. The following
>>> consuming route works fine:
>>>
>>> fromUri=sftp://myUser@myServer/subdir?password=myPassword&stepwise=false
>>>
>>> I think I had to use double "/" in Camel 2.10.3 but with Camel 2.11.0 it
>>> works with a single "/".
>>>
>>> Looking at the code and the full stacktrace (I include it at the end of
>>> the mail) it seems like what fails is the initial building of the
>>> subdirectory. In the process, Camel is changing directory down to the
>>> subdirectory and when done changes back to the starting/original directory.
>>> However, for some reason, Camel thinks that the original directory is "//"
>>> which is an incorrect path and fails.
>>>
>>> So it seems like it is the detection of the original directory that does
>>> not work. I currently have no workaround for this. If I skip using a sub
>>> directory it works. E g:
>>>
>>> sftp://myUser@myServer/?password=myPassword&stepwise=false
>>>
>>> In that case Camel does not need to change directory to the subdirectory
>>> and therefore does not need to change back either. Unfortunately I am not
>>> in control of the directory structure. I need to be able to poll files from
>>> one subdirectory (which works) and write files to another subdirectory
>>> (which doesn't work).
>>>
>>> Here is the stacktrace:
>>>
>>> 2013-04-24 08:34:38,533 | ERROR | terfaces/nfm/out | DefaultErrorHandler
>>>              | rg.apache.camel.util.CamelLogger  215 | Failed delivery for
>>> (MessageId: ID-IT-D-FQR815J-62818-1366717659819-18-1 on ExchangeId:
>>> ID-IT-D-FQR815J-62818-1366717659819-18-2). Exhausted after delivery
>>> attempt: 1 caught:
>>> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
>>> change directory to: //. Processed by failure processor:
>>> FatalFallbackErrorHandler[Channel[Wrap[se.digia.connect.service.filetransfer.FileTransferService$NotificationProcessor@2ba6ac4c]
>>> ->
>>> se.digia.connect.service.filetransfer.FileTransferService$NotificationProcessor@2ba6ac4c
>>> ]]
>>>  org.apache.camel.component.file.GenericFileOperationFailedException:
>>> Cannot change directory to: //
>>> at
>>> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:444)[126:org.apache.camel.camel-ftp:2.11.0]
>>>  at
>>> org.apache.camel.component.file.remote.SftpOperations.changeCurrentDirectory(SftpOperations.java:401)[126:org.apache.camel.camel-ftp:2.11.0]
>>> at
>>> org.apache.camel.component.file.remote.SftpOperations.buildDirectory(SftpOperations.java:340)[126:org.apache.camel.camel-ftp:2.11.0]
>>>  at
>>> org.apache.camel.component.file.GenericFileProducer.writeFile(GenericFileProducer.java:246)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.component.file.GenericFileProducer.processExchange(GenericFileProducer.java:159)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.component.file.remote.RemoteFileProducer.process(RemoteFileProducer.java:49)[126:org.apache.camel.camel-ftp:2.11.0]
>>> at
>>> org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.SendProcessor$2.doInAsyncProducer(SendProcessor.java:122)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.impl.ProducerCache.doInAsyncProducer(ProducerCache.java:298)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.SendProcessor.process(SendProcessor.java:117)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:72)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.interceptor.BacklogTracerInterceptor.process(BacklogTracerInterceptor.java:84)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.interceptor.TraceInterceptor.process(TraceInterceptor.java:91)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.RedeliveryErrorHandler.processErrorHandler(RedeliveryErrorHandler.java:390)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:273)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:46)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.interceptor.DefaultChannel.process(DefaultChannel.java:335)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.Pipeline.process(Pipeline.java:117)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.processor.Pipeline.process(Pipeline.java:80)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.RouteContextProcessor.processNext(RouteContextProcessor.java:46)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.UnitOfWorkProcessor.processAsync(UnitOfWorkProcessor.java:150)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.processor.UnitOfWorkProcessor.process(UnitOfWorkProcessor.java:117)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.RouteInflightRepositoryProcessor.processNext(RouteInflightRepositoryProcessor.java:48)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:73)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.processor.DelegateAsyncProcessor.processNext(DelegateAsyncProcessor.java:99)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.processor.DelegateAsyncProcessor.process(DelegateAsyncProcessor.java:90)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:72)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:350)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:197)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:163)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:141)[105:org.apache.camel.camel-core:2.11.0]
>>> at
>>> org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:91)[105:org.apache.camel.camel-core:2.11.0]
>>>  at
>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)[:1.6.0_32]
>>> at
>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)[:1.6.0_32]
>>>  at
>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)[:1.6.0_32]
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)[:1.6.0_32]
>>>  at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)[:1.6.0_32]
>>> at
>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)[:1.6.0_32]
>>>  at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)[:1.6.0_32]
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)[:1.6.0_32]
>>>  at java.lang.Thread.run(Thread.java:662)[:1.6.0_32]
>>> Caused by: 2: File not found
>>> at
>>> com.jcraft.jsch.ChannelSftp.throwStatusError(ChannelSftp.java:2629)[125:org.apache.servicemix.bundles.jsch:0.1.49.1]
>>>  at
>>> com.jcraft.jsch.ChannelSftp._stat(ChannelSftp.java:2078)[125:org.apache.servicemix.bundles.jsch:0.1.49.1]
>>> at com.jcraft.jsch.ChannelSftp.cd
>>> (ChannelSftp.java:316)[125:org.apache.servicemix.bundles.jsch:0.1.49.1]
>>>  at
>>> org.apache.camel.component.file.remote.SftpOperations.doChangeDirectory(SftpOperations.java:442)[126:org.apache.camel.camel-ftp:2.11.0]
>>> ... 55 more
>>>
>>>
>>> /Bengt
>>>
>>>
>>>
>>> 2013/4/23 Bengt Rodehav <[hidden email]<UrlBlockedError.aspx>>
>>>
>>>> Thanks for both your answers Chris and mdo,
>>>>
>>>> I will try to move the username  as you both suggest but according to
>>>> the documentation this should still be valid syntax.
>>>>
>>>> What puzzles me is the "...Cannot change directory to: //" since it
>>>> sounds like Camel is actually trying to change directory despite the
>>>> "stepwise=false".
>>>>
>>>> Will try tomorrow when I'm back at work.
>>>>
>>>> Thanks,
>>>>
>>>> /Bengt
>>>>
>>>>
>>>> 2013/4/23 mdo <[hidden email]<UrlBlockedError.aspx>>
>>>>
>>>>>
>>>>> BTW: I'm using URIs like this successfully with 2.11:
>>>>> sftp://192.168.1.2:22//home/user/
>>>>> sftp://localhost//tmp/subdir/
>>>>>
>>>>> Both being absolute paths. Login name and passwords appended along with
>>>>> other options.
>>>>>
>>>>> Regards, mdo.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> View this message in context:
>>>>> http://camel.465427.n5.nabble.com/Problems-with-sftp-in-Camel-2-11-0-tp5731359p5731372.html
>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>
>>>>
>>>>
>>>
>>
>


________________________________
If you reply to this email, your message will be added to the discussion below:
http://camel.465427.n5.nabble.com/Problems-with-sftp-in-Camel-2-11-0-tp5731359p5731408.html
To start a new topic under Camel - Users, email ml-node+s465427n465428h15@n5.nabble.com
To unsubscribe from Camel - Users, click here<http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=465428&code=ZGlldGhlci53dXl0c0BvcHRpcy5iZXw0NjU0Mjh8OTE4NjI3OTg5>.
NAML<http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>




--
View this message in context: http://camel.465427.n5.nabble.com/Problems-with-sftp-in-Camel-2-11-0-tp5731359p5731410.html
Sent from the Camel - Users mailing list archive at Nabble.com.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message