camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chirag Dewan <chirag.dewa...@yahoo.in>
Subject Re: OOM issue due to MemoryIdempotentRepository
Date Wed, 12 Feb 2014 12:37:04 GMT
Hi Claus,

Thanks for the quick reply.

It might well be. But there is one thing I would like to get some insight into. 

If idempotent=false(default) is used,will the LRU Cache still store the file referrence? Because
that is what I can see in my memory dump. 

org.apache.camel.util.LRUCache
  >org.apache.camel.com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap

I maybe doing something wrong here. Is there anything else I need to do to make sure that
file is deleted after it has been processed and not stored in the cache?

Thanks.

Chirag Dewan





________________________________
 From: Claus Ibsen <claus.ibsen@gmail.com>
To: "users@camel.apache.org" <users@camel.apache.org> 
Sent: Wednesday, 12 February 2014 5:42 PM
Subject: Re: OOM issue due to MemoryIdempotentRepository
 

Hi

Some days ago someone else posted about a OOME issue when using Storm.
It smells like a Storm + Camel issue somewhere.



On Wed, Feb 12, 2014 at 11:50 AM, Chirag Dewan <chirag.dewan22@yahoo.in> wrote:
> Hi All,
>
> I am using Camel 2.12.1 with JDK 1.7.0_45. I have a FTP consumer route in my application.
>
>
> Now I intent to delete the file after consuming it. So I have delete=true in my route.
After consuming around 6000 files(out of 10000,as my test case) JVM goes Out of memory.
>
>
> In my heap dump I can see the following trace as the major memory consumers:
>
> java.util.ArrayList
>  >java.lang.Object
>
>     >org.apache.camel.management.DefaultManagementLifecycleStrategy$PreRegisterService
>
>         >org.apache.camel.component.file.FileEndpoint
>
>             >org.apache.camel.processor.idempotent.MemoryIdempotentRepository
>                  >org.apache.camel.util.LRUCache
>                     >org.apache.camel.com.googlecode.concurrentlinkedhashmap.ConcurrentLinkedHashMap
>
>
> So if I am not using indempotent,should it still store the files in the memory? Is this
memory repository used,even if I am deleting the files and not storing in memory? Is there
a way I can avoid that?
>
> I am testing this scenario under Storm as execution environment and with 2GB max heap
space.
>
> Thanks in advance
>
> Chirag Dewan



-- 
Claus Ibsen
-----------------
Red Hat, Inc.
Email: cibsen@redhat.com
Twitter: davsclaus
Blog: http://davsclaus.com
Author of Camel in Action: http://www.manning.com/ibsen
Make your Camel applications look hawt, try: http://hawt.io
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message