activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: out of memory using producer flow control and fileQueueCursor
Date Fri, 24 Feb 2012 11:52:37 GMT
That seems to be a reasonable expectation, that transaction size is
not limited to available memory.
There was some work on trunk to have an in-memory send transaction
overflow to a temp file but it looks
like the pending massages cursor is a limitation for the receive side.

If you can produce with a simple junit test case and open an issue, it
would be a good start point for further investigation.

Is the the behavior the same when u use the default store cursor? My
guess is that it is but just incase.

On 24 February 2012 00:47, mserrano <martin@attivio.com> wrote:
> version 5.5.1
> broker cfg:  memory: 256M, Store:10g, Swap/tmp:10g, persistence=true,
> producerFlowControl=true
>
> I'm using the  http://activemq.apache.org/message-cursors.html
> fileQueueCursor  destination policy for a queue.  It appears to properly
> page in messages in the cursor without blocking when it gets to 70% memory
> use.  The queue is being read by a transacted session consumer which is
> keeping open the transaction for a many messages (>10k).  The messages
> themselves are large (800k or so).  What I see is that the Queue
> pagedInMessages data structure will grow without bound and contains a
> reference to every message dispatched to the consumer.   Eventually the
> broker will run out of memory.
>
> Producer flow control does not get triggered (but if it did my transaction
> would not be able to get all the messages it needs and would never
> complete).
>
> Is it just not possible to use ActiveMQ in this way?  What I mean by this is
> that I'd like the total size of the number of messages * the size of the
> messages of an open transaction to be bounded by the disk store limits not
> by memory limits.  Since the messages are persisted, it seems like this
> should be possible.
>
> Any suggestions for what I should do differently?  Would the
> connection.setTransactedIndividualAck setting in 5.6 help here?
>
> I'm happy to work on fixes/tests for this but need some guidance.
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/out-of-memory-using-producer-flow-control-and-fileQueueCursor-tp4415752p4415752.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.



-- 
http://fusesource.com
http://blog.garytully.com

Mime
View raw message