activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <>
Subject Re: out of memory using producer flow control and fileQueueCursor
Date Fri, 24 Feb 2012 12:36:24 GMT
>From a fix perspective, have a look at

it seems to be what is needed but I don't think it is currently
utilised to the full or fleshed out.

On 24 February 2012 11:52, Gary Tully <> wrote:
> That seems to be a reasonable expectation, that transaction size is
> not limited to available memory.
> There was some work on trunk to have an in-memory send transaction
> overflow to a temp file but it looks
> like the pending massages cursor is a limitation for the receive side.
> If you can produce with a simple junit test case and open an issue, it
> would be a good start point for further investigation.
> Is the the behavior the same when u use the default store cursor? My
> guess is that it is but just incase.
> On 24 February 2012 00:47, mserrano <> wrote:
>> version 5.5.1
>> broker cfg:  memory: 256M, Store:10g, Swap/tmp:10g, persistence=true,
>> producerFlowControl=true
>> I'm using the
>> fileQueueCursor  destination policy for a queue.  It appears to properly
>> page in messages in the cursor without blocking when it gets to 70% memory
>> use.  The queue is being read by a transacted session consumer which is
>> keeping open the transaction for a many messages (>10k).  The messages
>> themselves are large (800k or so).  What I see is that the Queue
>> pagedInMessages data structure will grow without bound and contains a
>> reference to every message dispatched to the consumer.   Eventually the
>> broker will run out of memory.
>> Producer flow control does not get triggered (but if it did my transaction
>> would not be able to get all the messages it needs and would never
>> complete).
>> Is it just not possible to use ActiveMQ in this way?  What I mean by this is
>> that I'd like the total size of the number of messages * the size of the
>> messages of an open transaction to be bounded by the disk store limits not
>> by memory limits.  Since the messages are persisted, it seems like this
>> should be possible.
>> Any suggestions for what I should do differently?  Would the
>> connection.setTransactedIndividualAck setting in 5.6 help here?
>> I'm happy to work on fixes/tests for this but need some guidance.
>> --
>> View this message in context:
>> Sent from the ActiveMQ - User mailing list archive at
> --


View raw message