camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Niels <ni...@kabisa.nl>
Subject Failing FTP Component
Date Thu, 05 Jul 2012 13:41:50 GMT
Hey everybody,

I made wrote a little piece of camel to consume a ftp server.

But after it was running for some time, it throws an exception, keeps
running but doesn't consume anything any more. Also when I start it again
and there are a larger number of file waiting to be consumed it will crash
again. I already added an exception handler but it seems like is doesn't
catch the exceptions.

This is the exception I receive: 

Caused by:
[org.apache.camel.component.file.GenericFileOperationFailedException - File
operation failed: 150 Opening ASCII mode data connection for
2386442.XML(3895 bytes).
 Accept timed out. Code: 150]
org.apache.camel.component.file.GenericFileOperationFailedException: File
operation failed: 150 Opening ASCII mode data connection for
2386442.XML(3895 bytes).
 Accept timed out. Code: 150
	at
org.apache.camel.component.file.remote.FtpOperations.retrieveFileToStreamInBody(FtpOperations.java:336)
	at
org.apache.camel.component.file.remote.FtpOperations.retrieveFile(FtpOperations.java:297)
	at
org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:333)
	at
org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:94)
	at
org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:175)
	at
org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:136)
	at
org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:140)
	at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:92)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
	at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.SocketTimeoutException: Accept timed out
	at java.net.PlainSocketImpl.socketAccept(Native Method)
	at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:408)
	at java.net.ServerSocket.implAccept(ServerSocket.java:462)
	at java.net.ServerSocket.accept(ServerSocket.java:430)
	at
org.apache.commons.net.ftp.FTPClient._openDataConnection_(FTPClient.java:560)
	at org.apache.commons.net.ftp.FTPClient.retrieveFile(FTPClient.java:1442)
	at
org.apache.camel.component.file.remote.FtpOperations.retrieveFileToStreamInBody(FtpOperations.java:328)
	... 16 more
Caused by:
[org.apache.camel.component.file.GenericFileOperationFailedException -
Cannot retrieve file: GenericFile[2386448.XML] from:
Endpoint[ftp://1.1.1.1?delay=15000&delete=true&disconnect=true&exclude=((?i).*pdf$)&password=******&username=user]
org.apache.camel.component.file.GenericFileOperationFailedException: Cannot
retrieve file: GenericFile[2386448.XML] from:
Endpoint[ftp://1.1.1.1?delay=15000&delete=true&disconnect=true&exclude=((?i).*pdf$)&password=******&username=user]
	at
org.apache.camel.component.file.GenericFileConsumer.processExchange(GenericFileConsumer.java:338)
	at
org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(RemoteFileConsumer.java:94)
	at
org.apache.camel.component.file.GenericFileConsumer.processBatch(GenericFileConsumer.java:175)
	at
org.apache.camel.component.file.GenericFileConsumer.poll(GenericFileConsumer.java:136)
	at
org.apache.camel.impl.ScheduledPollConsumer.doRun(ScheduledPollConsumer.java:140)
	at
org.apache.camel.impl.ScheduledPollConsumer.run(ScheduledPollConsumer.java:92)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
	at
java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317)
	at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180)
	at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
	at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
	at java.lang.Thread.run(Thread.java:662)

And this is the route I made using the Java DSL:

// XML Predicate
		// only allows names without spaces
		Predicate xmlPredicate = header(RssUtils.CAMEL_FILE_NAME).regex(
				"([\\S]+(\\.(?i)(xml))$)");
		// Images Predicate
		// only allows names without spaces
		Predicate imgPredicate = header(RssUtils.CAMEL_FILE_NAME).regex(
				"([\\S]+(\\.(?i)(jpg|png|gif))$)");

		onException(SchemaValidationException.class).to(
				"file://" + props.getProperty(RssUtils.ROOT_DIR)
						+ "/errors/SchemaValidationException");

		onException(GenericFileOperationFailedException.class).to(
				"file://" + props.getProperty(RssUtils.ROOT_DIR)
						+ "/errors/GenericFileExceptions");

		from(
				"ftp://"
						+ props.getProperty(RssUtils.FTP_URL)
						+ "?username="
						+ props.getProperty(RssUtils.FTP_USER)
						+ "&password="
						+ props.getProperty(RssUtils.FTP_PWD)
						+ "&disconnect=true&delete=true&exclude=((?i).*pdf$)&delay="
						+ props.getProperty(RssUtils.FTP_DELAY))
				.choice()
				.when(xmlPredicate)
				.to("jms:xmlQueue")
        .to("jms:archiveQueue")
				.when(imgPredicate)
				.to("file://" + props.getProperty(RssUtils.ROOT_DIR) + "/img")
				.otherwise()
				.to("file://" + props.getProperty(RssUtils.ROOT_DIR)
						+ "/errors/other");

		from("jms:xmlQueue").to("validator:FtpXmlValidator.xsd")
				.to("xslt://XmlToRssConverter.xsl")
				.process(rssFeedProcessor)
				.to("file://" + props.getProperty(RssUtils.ROOT_DIR) + "/rss/");

    from("jms:archiveQueue")
        .to("file://" + props.getProperty(RssUtils.ROOT_DIR) + "/archive/");

Is there anything I can do to avoid this kind af behavior? It is really
difficult to test so I'm hoping somebody spots a flaw in my code. I have
searching for quite some time now but I don't find anything solid.

There maybe a few things that I found somebody could give his tought on:

- use handled(true) when using the onException
- can I set the max batch size of the consumer? (can I use throttle for this
?)
- use explicit try catch finally because I'm using the Java DSL

Don't shoot me if I'm saying anything wrong here, I just learning Camel.
So if anybody has suggestions on the code above I would appreciate it!

Thanks a lot in advance!

--
View this message in context: http://camel.465427.n5.nabble.com/Failing-FTP-Component-tp5715550.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Mime
View raw message