camel-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Klaus Johansen (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CAMEL-10440) ftp component: Errors during file download bypass redelivery and goes directly to the deadletterchannel
Date Fri, 06 Jan 2017 19:15:58 GMT

    [ https://issues.apache.org/jira/browse/CAMEL-10440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15805342#comment-15805342
] 

Klaus Johansen edited comment on CAMEL-10440 at 1/6/17 7:15 PM:
----------------------------------------------------------------

Thanks to Lasse Lindgård we now have a functional work around: 

- Use consumer.bridgeErrorHandler=true 
- Apply custom PollingConsumerPollingStrategy: pollStrategy=#pollStrategyBean
- Make the custom PollingConsumerPollingStrategy stateful to record if socket timeout occurred
(per endpoint)
- Always clear recorded endpoint state in pollStategy begin() 
- Catch SocketTimeoutException and record the endpoint error state in the pollStategy bean,
e.g.
  ** implement Processor interface in pollStrategy and call the process()-method via onException(SocketTimeoutException.class).process("pollStrategyBean")
)
  ** or
  ** call a method (return true/false) directly from onWhen (provides some flexibility): onException(SocketTimeoutException.class).onWhen(simple("${bean:
pollStrategyBean.markError")) 
- If an timeout error was recorded a new exception can be thrown in commit() (on exhaustion).
This causes rollback() and appropriate disconnect and retry code can be applied.

I tend to believe that such SocketTimeoutExceptions should be handled "inside" the FTP consumer.


Best regards,
Klaus
 


was (Author: kluggi):
Thanks to Lasse Lindgård we now have a functional work around: 

- Use consumer.bridgeErrorHandler=true 
- Apply custom PollingConsumerPollingStrategy: pollStrategy=#pollStrategyBean
- Make the custom PollingConsumerPollingStrategy stateful to record if socket timeout occurred
(per endpoint)
- Always clear recorded endpoint state in pollStategy begin() 
- Catch SocketTimeoutException and record the endpoint error state in the pollStategy bean,
e.g.
  ** implement Processor interface in pollStrategy and call the process()-method via onException(SocketTimeoutException.class).process("pollStrategyBean")
)
  ** or
  ** call a method (return true/false) directly from onWhen (provides some flexibility): onException(SocketTimeoutException.class).onWhen(simple("${bean:
pollStrategyBean.markError")) 
- If an timeout error was recorded a new exception can be thrown in commit() (on exhaustion).
This causes rollback() and appropriate disconnect and retry code can be applied.

I tend to believe that such SocketTimeoutExceptions should be handled "inside" the FTP consumer.

 

> ftp component: Errors during file download bypass redelivery and goes directly to the
deadletterchannel 
> --------------------------------------------------------------------------------------------------------
>
>                 Key: CAMEL-10440
>                 URL: https://issues.apache.org/jira/browse/CAMEL-10440
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-ftp
>    Affects Versions: 2.17.3, 2.18.0
>         Environment: Behavior verified on MS Windows 7 (Spring Boot based application)
and MacOS (set based on "camel-example-ftp"). 
>            Reporter: Klaus Johansen
>            Priority: Minor
>         Attachments: ConsumerPollStrategy.java, MyFtpServer.java, MyFtpServerRouteBuilder.java
>
>
> When the FTP consumer is hit by an error during download of a file (eg. socket timeout
or because of missing read permission on the file) a GenericFileOperationFailedException is
thrown. Unfortunately is impossible to recover from this error using redelivery because the
message is forced onto the deadletterchannel (marked exhausted?)
> When using consumer.bridgeErrorHandler=true redelivery works fine when the errors happens
during the polling phase (eg. connection timeouts and missing directories).  
> When the error goes directly to the deadletterchannel the rollback method of the PollingConsumerPollStrategy
is never called. This makes it difficult to access the consumer and force a disconnect in
order to recover from the error situtation (like it is done by RemoteFilePollingConsumerPollStrategy).

> [DefaultQuartzScheduler-camel-1_Worker-1] INFO org.apache.camel.example.ftp.ConsumerPollStrategy
- Poll starting for endpoint: ftp://localhost:21/testdir?autoCreate=false&consumer.bridgeErrorHandler=true&fil...
> [DefaultQuartzScheduler-camel-1_Worker-1] DEBUG org.apache.camel.component.file.remote.FtpConsumer
- Exception checking connection status: File operation failed: null Connection is not open.
Code: 0
> [DefaultQuartzScheduler-camel-1_Worker-1] DEBUG org.apache.camel.component.file.remote.FtpConsumer
- Not connected/logged in, connecting to: ftp://testuser@localhost:21
> [DefaultQuartzScheduler-camel-1_Worker-1] DEBUG org.apache.camel.component.file.remote.FtpConsumer
- Connected and logged in to: ftp://testuser@localhost:21
> [DefaultQuartzScheduler-camel-1_Worker-1] DEBUG org.apache.camel.component.file.remote.FtpConsumer
- Took 0.151 seconds to poll: testdir
> [DefaultQuartzScheduler-camel-1_Worker-1] DEBUG org.apache.camel.component.file.remote.FtpConsumer
- Total 1 files to consume
> [DefaultQuartzScheduler-camel-1_Worker-1] DEBUG org.apache.camel.processor.DeadLetterChannel
- Failed delivery for (MessageId: ID-Klauss-MacBook-Pro-2-local-52649-1478165172245-0-2 on
ExchangeId: ID-Klauss-MacBook-Pro-2-local-52649-1478165172245-0-3). On delivery attempt: 0
caught: org.apache.camel.component.file.GenericFileOperationFailedException: Cannot retrieve
file: RemoteFile[testfile.txt] from: ftp://localhost:21/testdir?autoCreate=false&consumer.bridgeErrorHandler=true&fileName=testfile.txt&password=xxxxxx&pollStrategy=%23try3times&processStrategy=%23myProcessStrategy&scheduler=quartz2&scheduler.cron=15+0%2F1+*+%3F+*+*&throwExceptionOnConnectFailed=true&timeout=5000&username=testuser
> org.apache.camel.component.file.GenericFileOperationFailedException: Cannot retrieve
file: RemoteFile[testfile.txt] from: ftp://localhost:21/testdir?autoCreate=false&consumer.bridgeErrorHandler=true&fileName=testfile.txt&password=xxxxxx&pollStrategy=%23try3times&processStrategy=%23myProcessStrategy&scheduler=quartz2&scheduler.cron=15+0%2F1+*+%3F+*+*&throwExceptionOnConnectFailed=true&timeout=5000&username=testuser
> 	at org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:436)
> 	at org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:137)
> 	at org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:227)
> 	at org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:191)
> 	at org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:175)
> 	at org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:102)
> 	at org.apache.camel.pollconsumer.quartz2.QuartzScheduledPollConsumerJob.execute(QuartzScheduledPollConsumerJob.java:61)
> 	at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
> 	at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
> [DefaultQuartzScheduler-camel-1_Worker-1] DEBUG org.apache.camel.processor.SendProcessor
- >>>> direct://deadletter Exchange[ID-Klauss-MacBook-Pro-2-local-52649-1478165172245-0-3]
> [DefaultQuartzScheduler-camel-1_Worker-1] INFO org.apache.camel.processor.interceptor.Tracer
- ID-Klauss-MacBook-Pro-2-local-52649-1478165172245-0-3 >>> (route1) from(ftp://localhost:21/testdir?autoCreate=false&consumer.bridgeErrorHandler=true&fileName=testfile.txt&password=testpass&pollStrategy=%23try3times&processStrategy=%23myProcessStrategy&scheduler=quartz2&scheduler.cron=15+0%2F1+*+%3F+*+*&throwExceptionOnConnectFailed=true&timeout=5000&username=testuser)
--> log[On dead letter!!] <<< Pattern:InOnly, Headers:{breadcrumbId=ID-Klauss-MacBook-Pro-2-local-52649-1478165172245-0-2},
BodyType:String, Body:Error processing file RemoteFile[testfile.txt] due to Cannot retrieve
file: RemoteFile[testfile.txt] from: ftp://localhost:21/testdir?autoCreate=false&consumer.bridgeErrorHandler=true&fileName=testfile.txt&password=xxxxxx&pollStrategy=%23try3times&processStrategy=%23myProcessStrategy&scheduler=quartz2&scheduler.cron=15+0%2F1+*+%3F+*+*&throwExceptionOnConnectFailed=true&timeout=5000&username=testuser
> [DefaultQuartzScheduler-camel-1_Worker-1] INFO route1 - On dead letter!!
> [DefaultQuartzScheduler-camel-1_Worker-1] DEBUG org.apache.camel.processor.Pipeline -
Message exchange has failed: so breaking out of pipeline for exchange: Exchange[ID-Klauss-MacBook-Pro-2-local-52649-1478165172245-0-3]
Handled by the error handler.
> [DefaultQuartzScheduler-camel-1_Worker-1] INFO org.apache.camel.example.ftp.ConsumerPollStrategy
- Commiting 1 msgs for endpoint: ftp://localhost:21/testdir?autoCreate=false&consumer.bridgeErrorHandler=true&fil...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message