camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ranx <brad.john...@mediadriver.com>
Subject Re: Best Strategy to process a large number of rows in File
Date Tue, 29 Mar 2016 18:11:21 GMT
Jens,

That's why I suggested setting the limit on the queue size.  She has
streaming turned on already so I believe that will block when the queue
(SEDA or JMS) gets full.  But 50,000 objects isn't usually that much memory
so there may be something else in the JMS settings that is actually
marshalling and unmarshalling all the data. It may not be a local in memory
queue.  That would end up duplicating everything.  And it appears that the
out of memory is associated with AMQ itself.  Without digging into that can
of worms immediately it is easy enough to switch over to SEDA just to get
the functionality working and throughput parameters established and then
take a look at what might be happening with JMS. 

The SEDA queue has a "blockWhenFull" setting that should make that
relatively easy.  It's hard to say whether the overhead of JMS is necessary
in this case without knowing the actual business case and transactional
integrity requirements.



--
View this message in context: http://camel.465427.n5.nabble.com/Best-Strategy-to-process-a-large-number-of-rows-in-File-tp5779856p5779989.html
Sent from the Camel - Users mailing list archive at Nabble.com.

Mime
View raw message