Return-Path: X-Original-To: apmail-camel-dev-archive@www.apache.org Delivered-To: apmail-camel-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 6A43773CC for ; Thu, 3 Nov 2011 13:01:57 +0000 (UTC) Received: (qmail 69876 invoked by uid 500); 3 Nov 2011 13:01:57 -0000 Delivered-To: apmail-camel-dev-archive@camel.apache.org Received: (qmail 69848 invoked by uid 500); 3 Nov 2011 13:01:57 -0000 Mailing-List: contact dev-help@camel.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@camel.apache.org Delivered-To: mailing list dev@camel.apache.org Received: (qmail 69840 invoked by uid 99); 3 Nov 2011 13:01:57 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Nov 2011 13:01:57 +0000 X-ASF-Spam-Status: No, hits=-2001.2 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 03 Nov 2011 13:01:53 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id 7E25132F037 for ; Thu, 3 Nov 2011 13:01:32 +0000 (UTC) Date: Thu, 3 Nov 2011 13:01:32 +0000 (UTC) From: "Aleksey Sushko (Commented) (JIRA)" To: dev@camel.apache.org Message-ID: <464354268.55526.1320325292518.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <1767779107.49652.1320218492918.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Commented] (CAMEL-4605) Camel ftp read file after close connection MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/CAMEL-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13143110#comment-13143110 ] Aleksey Sushko commented on CAMEL-4605: --------------------------------------- Redefine the creation of a method of FTPClient FTPClient createFtpClient () class FtpEndpoint. Add to the method displays the message {code} LOGGER.info("create FTPClient {}", client); {code} {code} public class FTPClientTest extends FTPClient { private static final Logger LOGGER = LoggerFactory.getLogger(FTPClientTest.class); public FTPClientTest() { super(); } @Override public void disconnect() throws IOException { LOGGER.info("FTPClient disconnect {}", this); super.disconnect(); } @Override protected Socket _openDataConnection_(int command, String arg) throws IOException { LOGGER.info("FTPClient _openDataConnection_ {}", this); return super._openDataConnection_(command, arg); } @Override protected void _connectAction_() throws IOException { LOGGER.info("FTPClient _connectAction_ {}", this); super._connectAction_(); } } {code} We observe the creation of two FTPClient {code} 2011-11-03 15:22:00,862 | INFO | Blueprint Extender: 2 | FtpEndpointExt | t.file.remote.ext.FtpEndpointExt 55 | 115 - org.apache.camel.c amel-ftp - 2.8.1 | create FTPClient org.apache.camel.component.file.remote.ext.FTPClientTest@2e81d81e {code} ... {code} 2011-11-03 15:22:01,213 | INFO | Blueprint Extender: 1 | FtpEndpointExt | t.file.remote.ext.FtpEndpointExt 55 | 115 - org.apache.camel.c amel-ftp - 2.8.1 | create FTPClient org.apache.camel.component.file.remote.ext.FTPClientTest@738e7a09 {code} Last post by FTPClient {code} 2011-11-03 15:28:01,543 | INFO | Camel (exchange-read) thread #0 - ftp://mail01.zakupki.gov.ru/auto/organization/all/ | FTPClientTest| nt.file.remote.ext.FTPClientTest 25 | 115 - org.apache.camel.camel-ftp - 2.8.1 | FTPClient _openDataConnection_ org.apache.camel.component.file.remote.ext.FTPClientTest@2e81d81e {code} Note the number of object *FTPClientTest@2e81d81e*. Begins to stop the second process (thread # 1): {code} 2011-11-03 15:28:09,010 | INFO | Camel (exchange-read) thread #1 - ShutdownTask | DefaultShutdownStrategy | ultShutdownStrategy$ShutdownTask 425 | 94 - org.apache.camel.camel-core - 2.8.1 | Waiting as there are still 56 inflight and pending exchanges to complete, timeout in 291 seco nds. 2011-11-03 15:28:10,004 | WARN | DefaultQuartzScheduler-exchange-read_Worker-1 | DefaultShutdownStrategy | mel.impl.DefaultShutdownStrategy 143 | 94 - org.apache.camel.camel-core - 2.8.1 | Timeout occurred. Now forcing the routes to be shutdown now. 2011-11-03 15:28:10,004 | WARN | Camel (exchange-read) thread #1 - ShutdownTask | DefaultShutdownStrategy | ultShutdownStrategy$ShutdownTask 433 | 94 - org.apache.camel.camel-core - 2.8.1 | Interrupted while waiting during graceful shutdown, will force shutdown now. {code} Completing the process (thread # 0): {code} 2011-11-03 15:28:10,011 | TRACE | DefaultQuartzScheduler-exchange-read_Worker-1 | ServiceHelper | .apache.camel.util.ServiceHelper 103 | 94 - org.apache.camel.camel-core - 2.8.1 | Service already stopped: org.apache.camel.impl.SharedProducerServicePool@50ec2522 2011-11-03 15:28:10,011 | TRACE | DefaultQuartzScheduler-exchange-read_Worker-1 | ServiceHelper | .apache.camel.util.ServiceHelper 108 | 94 - org.apache.camel.camel-core - 2.8.1 | Stopping service DefaultErrorHandler[TraceInterceptor[To[bean:dictStoreProcess]]] 2011-11-03 15:28:10,011 | DEBUG | DefaultQuartzScheduler-exchange-read_Worker-1 | FtpConsumer | t.file.remote.RemoteFileConsumer 102 | 94 - org.apache.camel.camel-core - 2.8.1 | Disconnecting from: ftp://anonymous@mail01.zakupki.gov.ru:21 2011-11-03 15:28:12,908 | INFO | DefaultQuartzScheduler-exchange-read_Worker-1 | FTPClientTest | nt.file.remote.ext.FTPClientTest 19 | 115 - org.apache.camel.camel-ftp - 2.8.1 | FTPClient disconnect org.apache.camel.component.file.remote.ext.FTPClientTest@2e81d81e {code} As you can see, the system is stopped *FTPClientTest@2e81d81e*. This object belongs to the first thread (thread # 0). > Camel ftp read file after close connection > ------------------------------------------ > > Key: CAMEL-4605 > URL: https://issues.apache.org/jira/browse/CAMEL-4605 > Project: Camel > Issue Type: Bug > Components: camel-ftp > Affects Versions: 2.8.1 > Reporter: Aleksey Sushko > > There is an error with ftp server. > Many files on the server. All files have no time to be processed. > When the processing of the next file, the process stops. > Error - The request is a file after closing the connection. > Camel config route: > {code} > > xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0" > xmlns:camel="http://camel.apache.org/schema/blueprint" > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" > default-activation="lazy"> > > > > > > > > > class="org.apache.camel.routepolicy.quartz.CronScheduledRoutePolicy"> > > > > xmlns="http://camel.apache.org/schema/blueprint" autoStartup="false"> > > poolSize="2" maxPoolSize="2" maxQueueSize="-1"/> > > routePolicyRef="routePolicy" > shutdownRunningTask= "CompleteAllTasks"> > > > > > > > {code} > We have two streams > bq. > The first thread (thread#0) reading file from ftp server > {code} > 2011-11-02 09:13:57,376 | TRACE | Camel (exchange-read) thread #0 - ftp://mail01.zakupki.gov.ru/auto/organization/all/ | FtpConsumer | mponent.file.GenericFileConsumer 261 | 94 - org.apache.camel.camel-core - 2.8.1 | > Processing file: GenericFile[organization_all_20111023_010003_33.xml.zip] > 2011-11-02 09:13:57,376 | TRACE | Camel (exchange-read) thread #0 - ftp://mail01.zakupki.gov.ru/auto/organization/all/ | FtpConsumer | mponent.file.GenericFileConsumer 293 | 94 - org.apache.camel.camel-core - 2.8.1 | > Retrieving file: auto/organization/all/organization_all_20111023_010003_33.xml.zip from: Endpoint[ftp://mail01.zakupki.gov.ru/auto/organization/all/? > binary=true&connectTimeout=10000&delay=600000&disconnect=true&localWorkDirectory=%2Fhome%2Fuser%2Ftemp&noop=true&passiveMode=true&soTimeout=10000&timeout=15000] > {code} > The second stream (thread#1) is stopped > {code} > 2011-11-02 09:14:00,004 | DEBUG | Camel (exchange-read) thread #1 - ShutdownTask | ScheduledPollConsumer | camel.impl.ScheduledPollConsumer 265 | 94 - org.apache.camel.camel-core - 2.8.1 | > This consumer is stopping, so cancelling scheduled task: java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@2bf8d0a4 > 2011-11-02 09:14:00,005 | DEBUG | Camel (exchange-read) thread #1 - ShutdownTask | FtpConsumer | pache.camel.impl.DefaultConsumer 83 | 94 - org.apache.camel.camel-core - 2.8.1 | > Stopping consumer: FtpConsumer[ftp://mail01.zakupki.gov.ru/auto/organization/all/?binary=true&connectTimeout=10000&delay=600000&disconnect=true&localWorkDirectory=%2Fhome%2Fuser%2Ftemp&noop=true&passiveMode=true&soTimeout=10000&timeout=15000] > 2011-11-02 09:14:00,007 | DEBUG | Camel (exchange-read) thread #1 - ShutdownTask | ProcessorEndpoint$1 | pache.camel.impl.DefaultProducer 76 | 94 - org.apache.camel.camel-core - 2.8.1 | > Stopping producer: Producer[bean://readProcess] > 2011-11-02 09:14:00,010 | DEBUG | Camel (exchange-read) thread #1 - ShutdownTask | FtpConsumer | t.file.remote.RemoteFileConsumer 102 | 94 - org.apache.camel.camel-core - 2.8.1 | > Disconnecting from: ftp://anonymous@mail01.zakupki.gov.ru:21 > {code} > Warning!!!! ftp close connection. Completed method FTPClient.disconnect(). Parament _controlInput_ = null. > {code} > 2011-11-02 09:14:03,182 | INFO | Camel (exchange-read) thread #1 - ShutdownTask | DefaultShutdownStrategy | ultShutdownStrategy$ShutdownTask 460 | 94 - org.apache.camel.camel-core - 2.8.1 | > Route: readRoute shutdown complete, was consuming from: Endpoint[...] > 2011-11-02 09:14:03,182 | INFO | DefaultQuartzScheduler-exchange-read_Worker-1 | DefaultShutdownStrategy | mel.impl.DefaultShutdownStrategy 158 | 94 - org.apache.camel.camel-core - 2.8.1 | > Graceful shutdown of 1 routes completed in 3 seconds > {code} > The first thread (thread#0) continues reading the file [organization_all_20111023_010003_33.xml.zip]. > {code} > 2011-11-02 09:14:04,172 | ERROR | Camel (exchange-read) thread #0 - ftp://mail01.zakupki.gov.ru/auto/organization/all/ | FtpConsumer | ache.camel.processor.CamelLogger 232 | 94 - org.apache.camel.camel-core - 2.8.1 | Caused by: [java.lang.NullPointerException - null] > java.lang.NullPointerException > at org.apache.commons.net.ftp.FTP.__getReply(FTP.java:295)[97:org.apache.commons.net:2.2] > at org.apache.commons.net.ftp.FTP.getReply(FTP.java:622)[97:org.apache.commons.net:2.2] > at org.apache.commons.net.ftp.FTPClient.completePendingCommand(FTPClient.java:1408)[97:org.apache.commons.net:2.2] > at org.apache.commons.net.ftp.FTPClient.retrieveFile(FTPClient.java:1467)[97:org.apache.commons.net:2.2] > at org.apache.camel.component.file.remote.FtpOperations.retrieveFileToFileInLocalWorkDirectory(FtpOperations.java:406)[99:org.apache.camel.camel-ftp:2.8.1] > {code} -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira