camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Achim Nierbeck <bcanh...@googlemail.com>
Subject FTP download of bigger files gives me OOM
Date Thu, 25 Oct 2012 14:51:10 GMT
Hi,

using Camel 2.10.2
I have a ftp endpoint that downloads a couple of files from a remote
FTP server.
There are different files located on this server, some of them are
about 700 MB size.
Now every time I start the route for downloading it gives me a OOM
when trying to download this file.
The route is configured to use the *localWorkDirectory*.
Roughly the route does something like the following:

from("ftp://server?noop=true&localWorkDirectory=/myTemp&sendEmptyMessageWhenIdle=true")
.onException()
    ... ommited the exception part ....
.onCompletion()
    ... omitted the onCompletion part ....
.choice()
     .when(body().isNull())
         .... stop the whole thing here ...
     .otherwise()
         .to("file:/dropItHere")
          .setBody(constant(""))
          .to("seda:doSomeLogging");

I tried to increase the memory up to 3GB for the Heap size with no luck.

Following Stacktrace is logged:
Camel (rootContext) thread #28 - ftp://usrXXX@ServerXXXX/
  at java.lang.OutOfMemoryError.<init>()V (OutOfMemoryError.java:25)
  at java.util.Arrays.copyOf([BI)[B (Arrays.java:2786)
  at java.io.ByteArrayOutputStream.write([BII)V (ByteArrayOutputStream.java:94)
  at org.apache.commons.net.io.Util.copyStream(Ljava/io/InputStream;Ljava/io/OutputStream;IJLorg/apache/commons/net/io/CopyStreamListener;Z)J
(Util.java:123)
  at org.apache.commons.net.ftp.FTPClient._retrieveFile(Ljava/lang/String;Ljava/lang/String;Ljava/io/OutputStream;)Z
(FTPClient.java:1695)
  at org.apache.commons.net.ftp.FTPClient.retrieveFile(Ljava/lang/String;Ljava/io/OutputStream;)Z
(FTPClient.java:1669)
  at org.apache.camel.component.file.remote.FtpOperations.retrieveFileToStreamInBody(Ljava/lang/String;Lorg/apache/camel/Exchange;)Z
(FtpOperations.java:328)
  at org.apache.camel.component.file.remote.FtpOperations.retrieveFile(Ljava/lang/String;Lorg/apache/camel/Exchange;)Z
(FtpOperations.java:297)
  at org.apache.camel.component.file.GenericFileConsumer.processExchange(Lorg/apache/camel/Exchange;)V
(GenericFileConsumer.java:317)
  at org.apache.camel.component.file.remote.RemoteFileConsumer.processExchange(Lorg/apache/camel/Exchange;)V
(RemoteFileConsumer.java:94)
  at org.apache.camel.component.file.GenericFileConsumer.processBatch(Ljava/util/Queue;)I
(GenericFileConsumer.java:189)
  at org.apache.camel.component.file.GenericFileConsumer.poll()I
(GenericFileConsumer.java:155)
  at org.apache.camel.impl.ScheduledPollConsumer.doRun()V
(ScheduledPollConsumer.java:139)
  at org.apache.camel.impl.ScheduledPollConsumer.run()V
(ScheduledPollConsumer.java:91)


The attached OOM-Camel.PNG shows what the Memory Analysis tells me.

Am I missing something critical here? Or do I have to tune up the
memory for those kind
of cases?


Thanks and regards, Achim
-- 

Apache Karaf <http://karaf.apache.org/> Committer & PMC
OPS4J Pax Web <http://wiki.ops4j.org/display/paxweb/Pax+Web/>
Committer & Project Lead
OPS4J Pax for Vaadin
<http://team.ops4j.org/wiki/display/PAXVAADIN/Home> Commiter & Project
Lead
blog <http://notizblog.nierbeck.de/>

Mime
  • Unnamed multipart/mixed (inline, None, 0 bytes)
View raw message