activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabrice Delaporte (JIRA)" <>
Subject [jira] Closed: (AMQ-2532) Broker paging all available messages from store leading to resource exhaustion
Date Mon, 14 Dec 2009 17:20:52 GMT


Fabrice Delaporte closed AMQ-2532.

       Resolution: Fixed
    Fix Version/s: 5.4.0

This is already fixed.

> Broker paging all available messages from store leading to resource exhaustion
> ------------------------------------------------------------------------------
>                 Key: AMQ-2532
>                 URL:
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.3.0
>         Environment: Java 1.6
>            Reporter: Fabrice Delaporte
>             Fix For: 5.3.1, 5.4.0
> There seems to be a regression (works fine in 5.2.0) related to the fix for the following:
> I think the intent of the original code was to prevent uncontrolled paging of pending
messages. The doPageIn method is called in version 5.3.0 :
> - each time a message is acked/committed
> - each time the MemoryUsage is decreased
> As a result, if you create a queue with, say, 1 000 000 messages in it, then start a
consumer, the broker will try to page in all the messages from the store as consuming goes
on (with default configuration values, 200 paged in messages for one consumed message minimum
- I say minimum because if the MemoryUsage is decreased when you consume a message, then doPageIn
is invoked again). Guaranteed OutOfMemoryError as in general consuming messages is slower
than paging from store. I guess the same can occur when you have a fast producer and a slow
consumer, without producer flow control.
> I would say that in any case, the broker should never page in more messages than the
maxPageSize attribute, plus the number of dispatched messages. More formally:
> {code:java}
>             int toPageIn = getMaxPageSize() - pagedInPendingDispatch.size();
> {code}
> And just to make sure we never get a negative value:
> {code:java}
>             int toPageIn = Math.max(0, getMaxPageSize() - pagedInPendingDispatch.size());
> {code}
> Note 1: this behavior is also exhibited by the background message expiration logic, that
forces message pagination each time it is run. With big queues without consumers, by default
you'll get 200 more messages paginated each time the expiration task is run. Eventually leading
to broker resource exhaustion if no consumer comes into play in the near future with active
producers queuing up messages.
> Note 2: this may also be related to
> Workarounds:
> - setting lazyDispatch to true fixes the problem for message consuming only
> - background expiration can be disabled by setting expireMessagesPeriod to -1
> Cheers,
> Fabrice

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message