camel-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Claus Ibsen (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CAMEL-4605) Camel ftp read file after close connection
Date Fri, 04 Nov 2011 10:29:00 GMT

    [ https://issues.apache.org/jira/browse/CAMEL-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143894#comment-13143894
] 

Claus Ibsen commented on CAMEL-4605:
------------------------------------

The FTPClient library is not threads safe.

Where do you use that thread pool profile?
                
> Camel ftp read file after close connection
> ------------------------------------------
>
>                 Key: CAMEL-4605
>                 URL: https://issues.apache.org/jira/browse/CAMEL-4605
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-ftp
>    Affects Versions: 2.8.1
>            Reporter: Aleksey Sushko
>
> There is an error with ftp server. 
> Many files on the server. All files have no time to be processed. 
> When the processing of the next file, the process stops. 
> Error - The request is a file after closing the connection. 
> Camel config route: 
> {code}
> <?xml version="1.0" encoding="UTF-8"?>
> <blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
>   xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0"
>   xmlns:camel="http://camel.apache.org/schema/blueprint"
>   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>   default-activation="lazy">
>   <cm:property-placeholder persistent-id="ccwe.exchange.oos">
>     <cm:default-properties>
>       <cm:property name="ftp.doc.url" value="ftp://mail01.zakupki.gov.ru/auto/organization/all/"/>
>       <cm:property name="ftp.doc.add_params" value=""/>
>     </cm:default-properties>
>   </cm:property-placeholder>
>   
>   <bean id="readProcess" class="mypackage.docread.ReadProcess"/>
>   <bean id="routePolicy" 
>     class="org.apache.camel.routepolicy.quartz.CronScheduledRoutePolicy">
>     <property name="routeStartTime" value="* 0/10 * * * ?"/>
>     <property name="routeStopTime" value="* 4/10 * * * ?"/>
>   </bean>
>   <camelContext id="exchange-read" 
>      xmlns="http://camel.apache.org/schema/blueprint" autoStartup="false">
>      
>     <threadPoolProfile id="fooProfile" 
>        poolSize="2" maxPoolSize="2" maxQueueSize="-1"/>
>        
>     <route id="readRoute" 
>       routePolicyRef="routePolicy"
>       shutdownRunningTask= "CompleteAllTasks">
>       
>       <from uri="{{ftp.doc.url}}?noop=true&amp;binary=true{{ftp.doc.add_params}}"/>
>       <to uri="bean:readProcess"/>
>     </route>
>   </camelContext>
> </blueprint>
> {code}
> We have two streams
> bq. <threadPoolProfile id="fooProfile" poolSize="2" maxPoolSize="2" maxQueueSize="-1"/>
> The first thread (thread#0) reading file from ftp server
> {code}
> 2011-11-02 09:13:57,376 | TRACE | Camel (exchange-read) thread #0 - ftp://mail01.zakupki.gov.ru/auto/organization/all/
| FtpConsumer                      | mponent.file.GenericFileConsumer  261 | 94 - org.apache.camel.camel-core
- 2.8.1 |
>      Processing file: GenericFile[organization_all_20111023_010003_33.xml.zip]
> 2011-11-02 09:13:57,376 | TRACE | Camel (exchange-read) thread #0 - ftp://mail01.zakupki.gov.ru/auto/organization/all/
| FtpConsumer                      | mponent.file.GenericFileConsumer  293 | 94 - org.apache.camel.camel-core
- 2.8.1 |
>      Retrieving file: auto/organization/all/organization_all_20111023_010003_33.xml.zip
from: Endpoint[ftp://mail01.zakupki.gov.ru/auto/organization/all/?
> binary=true&connectTimeout=10000&delay=600000&disconnect=true&localWorkDirectory=%2Fhome%2Fuser%2Ftemp&noop=true&passiveMode=true&soTimeout=10000&timeout=15000]
> {code}
> The second stream (thread#1) is stopped
> {code}
> 2011-11-02 09:14:00,004 | DEBUG | Camel (exchange-read) thread #1 - ShutdownTask | ScheduledPollConsumer
           | camel.impl.ScheduledPollConsumer  265 | 94 - org.apache.camel.camel-core - 2.8.1
| 
>      This consumer is stopping, so cancelling scheduled task: java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@2bf8d0a4
> 2011-11-02 09:14:00,005 | DEBUG | Camel (exchange-read) thread #1 - ShutdownTask | FtpConsumer
                     | pache.camel.impl.DefaultConsumer   83 | 94 - org.apache.camel.camel-core
- 2.8.1 | 
>      Stopping consumer: FtpConsumer[ftp://mail01.zakupki.gov.ru/auto/organization/all/?binary=true&connectTimeout=10000&delay=600000&disconnect=true&localWorkDirectory=%2Fhome%2Fuser%2Ftemp&noop=true&passiveMode=true&soTimeout=10000&timeout=15000]
> 2011-11-02 09:14:00,007 | DEBUG | Camel (exchange-read) thread #1 - ShutdownTask | ProcessorEndpoint$1
             | pache.camel.impl.DefaultProducer   76 | 94 - org.apache.camel.camel-core -
2.8.1 | 
>      Stopping producer: Producer[bean://readProcess]
> 2011-11-02 09:14:00,010 | DEBUG | Camel (exchange-read) thread #1 - ShutdownTask | FtpConsumer
                     | t.file.remote.RemoteFileConsumer  102 | 94 - org.apache.camel.camel-core
- 2.8.1 | 
>      Disconnecting from: ftp://anonymous@mail01.zakupki.gov.ru:21
> {code}
> Warning!!!! ftp close connection. Completed method FTPClient.disconnect(). Parament _controlInput_
= null.
> {code}
> 2011-11-02 09:14:03,182 | INFO  | Camel (exchange-read) thread #1 - ShutdownTask | DefaultShutdownStrategy
         | ultShutdownStrategy$ShutdownTask  460 | 94 - org.apache.camel.camel-core - 2.8.1
| 
>      Route: readRoute shutdown complete, was consuming from: Endpoint[...]
> 2011-11-02 09:14:03,182 | INFO  | DefaultQuartzScheduler-exchange-read_Worker-1 | DefaultShutdownStrategy
         | mel.impl.DefaultShutdownStrategy  158 | 94 - org.apache.camel.camel-core - 2.8.1
| 
>      Graceful shutdown of 1 routes completed in 3 seconds
> {code}
> The first thread (thread#0) continues reading the file [organization_all_20111023_010003_33.xml.zip].
> {code}
> 2011-11-02 09:14:04,172 | ERROR | Camel (exchange-read) thread #0 - ftp://mail01.zakupki.gov.ru/auto/organization/all/
| FtpConsumer                      | ache.camel.processor.CamelLogger  232 | 94 - org.apache.camel.camel-core
- 2.8.1 | Caused by: [java.lang.NullPointerException - null]
> java.lang.NullPointerException
> 	at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:295)[97:org.apache.commons.net:2.2]
> 	at org.apache.commons.net.ftp.FTP.getReply(FTP.java:622)[97:org.apache.commons.net:2.2]
> 	at org.apache.commons.net.ftp.FTPClient.completePendingCommand(FTPClient.java:1408)[97:org.apache.commons.net:2.2]
> 	at org.apache.commons.net.ftp.FTPClient.retrieveFile(FTPClient.java:1467)[97:org.apache.commons.net:2.2]
> 	at org.apache.camel.component.file.remote.FtpOperations.retrieveFileToFileInLocalWorkDirectory(FtpOperations.java:406)[99:org.apache.camel.camel-ftp:2.8.1]
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message