activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From artnaseef <...@artnaseef.com>
Subject Re: Transactions and memory consumption
Date Fri, 11 Apr 2014 03:48:37 GMT
There is no way in ActiveMQ to clear those messages from memory while the
transaction is ongoing.  The data structure for the transaction holds the
message references.

Once in memory for a Queue, a message is only made available to the JVM GC
by one of the following:
* Acknowledgement of the message
* Expiration of the message
* Completion of a transaction consuming the message
* Administrative tools used to purge the Queue

I'm very curious about the use-case as the broker can hold a lot of message
in memory, given a good size JVM heap size.  What kind of performance
numbers are seen and how much improvement is expected by increasing the
numbers?  It should be possible to gauge the overall improvement by testing
with added Session.commit() calls periodically, or, for an XA-Transaction,
by using non-transacted Queue consumption.

If there is a long delay on committing downstream, perhaps a strategy to
improve throughput using parallel processing can work:
* Fill a transaction
* Start commit
* At the same time, continue consuming incoming messages into the next
transaction
* Fill second transaction
* Wait for first transaction commit to complete
* If first transaction failed, roll back the second
* If first transaction succeeded, commit the second and initiate the next
iteration of the same

Of course, in the long run, if the downstream system cannot keep up with the
rate of incoming messages, it will be impossible to prevent slow-consumption
problems from overloading the broker.

I'll post this on the stackoverflow question as well.



--
View this message in context: http://activemq.2283324.n4.nabble.com/Transactions-and-memory-consumption-tp4224862p4680214.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message