camel-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "bernard HAUZEUR (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CAMEL-10281) ProducerTemplate sending to variable URI's fills-up the heap
Date Thu, 01 Sep 2016 08:22:22 GMT

    [ https://issues.apache.org/jira/browse/CAMEL-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15454743#comment-15454743
] 

bernard HAUZEUR commented on CAMEL-10281:
-----------------------------------------

Hi Mr Ibsen, thanks for the speedy action but I can't agree. May be I was too verbose and
the following escaped to your attention. May be I should have gone via the Forum first and
let somebody else create the issue, humbly.

How do you explain that (excerpt):
"The ProducerTemplate is supposed to obey a max 1000 cache size default but ignores this boundary
and any other max cache size setting [...] e.g. trace producerTemplate.getCurrentCacheSize()"


and 
"[...] getContext().getEndpointMap(): [...] getContext().removeEndpoint(...) as obtained from
the previous map, the remove() generates no error, yet the producer template cache size does
not reduce" --> are we getting a copy of the actual endpoints? 
I even noted that I can remove a JMS consumer (static) endpoint (doc says it is stopped +
removed), yet traffic continues to flow in... (ok stopping the route does work)





> ProducerTemplate sending to variable URI's fills-up the heap
> ------------------------------------------------------------
>
>                 Key: CAMEL-10281
>                 URL: https://issues.apache.org/jira/browse/CAMEL-10281
>             Project: Camel
>          Issue Type: Bug
>          Components: camel-ftp
>    Affects Versions: 2.16.3
>         Environment: JDK 8, JBoss EAP7, WAR deployment with SpringFramework hosting CAMEL
context and routes, dynamic FTP sender to ad hoc (S)FTP destinations using ProducerTemplate
>            Reporter: bernard HAUZEUR
>            Assignee: Claus Ibsen
>              Labels: stability
>
> We have build a connector (WAR deployment) in XML DSL reading from a JMS queue, using
numerous custom processors, and sending messages to SFTP destination. The URI is built dynamically
and usually varies (file name, destination directory and server, etc) with each file being
transmitted.
> We conducted soak tests and the JVM becomes unresponsive after a few hours. The HEAP
increases steadily and rapidly to several Gbytes until we undeploy the WAR at which time the
GC can recover memory.
> We discovered that the memory actually fills up with dynamic endpoint instances. If we
trace producerTemplate.getCurrentCacheSize(), the number increases by 1 with every file being
transmitted. The ProducerTemplate is supposed to obey a max 1000 cache size default but ignores
this boundary and any other max cache size setting. 
> We can also trace this increasing count but this time up to 1000 max (+ number of static
endpoints) via getContext().getEndpointMap().size(). If we try compensating with getContext().removeEndpoint(...)
as obtained from the previous map, the remove() generates no error, yet the producer template
cache size does not reduce its ever increasing count.
> On the other hand, if we are careful to always guarantee that the CAMEL URI is identical
with every transmission and that all dynamic parameters are actually mapped on the exchange
header properties instead of CAMEL options, then the count of dynamic endpoints remains at
one from the start.
> Whoever builds URIs and not knowing this is at risk of breaking production after a few
hours or a few days according to traffic intensity.
> Links to CAMEL-2558, CAMEL-3827, CAMEL-7965



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message