activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jesus Roncero <je...@we7.com>
Subject Re: Understanding memoryLimits with flow control and systemUsage limits.
Date Thu, 06 Jun 2013 14:21:55 GMT
On Wed, Jun 05, 2013 at 09:03:15AM -0700, Christian Posta wrote:
> was just thinking... for question #3, it would make sense if the overall
> broker memory was 500MB and your temp was 100MB... then when you hit the
> cursor high water mark at some point (70%) it would spook to disk and
> overload the temp... in that case, you have to ensure the temp storage is
> sufficient to handle your messages from memory.... but as you described it
> (500mb per destination, 20mb overall) then it doesn't work.


Hi,

So, as I said, this happens, and I believe this config was copied from
somewhere. It definitely appears on google (i.e:
http://mail-archives.apache.org/mod_mbox/activemq-users/201001.mbox/%3C4B504637.3020306@oma.be%3E
) so, it might have something to do. 

The specific config I was talking about is:

  <policyEntries>
    <policyEntry queue=">" memoryLimit="500mb" producerFlowControl="false">
        <pendingQueuePolicy>
            <fileQueueCursor />
        </pendingQueuePolicy>
    </policyEntry>
    <policyEntry topic=">" memoryLimit="500mb" producerFlowControl="false">
        <!-- We want to store both persistent and non-persistent messages -->
        <pendingSubscriberPolicy>
            <fileCursor />
        </pendingSubscriberPolicy>
        <pendingDurableSubscriberPolicy>
            <fileDurableSubscriberCursor/>
        </pendingDurableSubscriberPolicy>

...

        <memoryUsage>
            <memoryUsage limit="20 mb"/>
        </memoryUsage>
        <storeUsage>
            <storeUsage limit="1 gb" />
        </storeUsage>
        <tempUsage>
            <tempUsage limit="100 mb"/>
        </tempUsage>

So, on this case (I know, the config is totally wrong, I'm just trying to
understand), when I insert 1Mb messages, it stops at 351 Mb (70% of 500Mb) and
then it gets a log like:

[2013-06-06 15:01:32,909] INFO BrokerService.worker.1 org.apache.activemq.store.kahadb.plist.PListStore
- PListStore:[/var/lib/activemq/we7/data/localhost/tmp_storage ] initialized 
2013-06-06 15:01:32,909 [ervice.worker.1] INFO  PListStore - PListStore:[/var/lib/activemq/we7/data/localhost/tmp_storage
] initialized 
[2013-06-06 15:01:52,141] INFO ActiveMQ Transport: tcp:///10.0.9.124:46281 org.apache.activemq.broker.region.Queue
- Usage(default:temp:queue://testqueue:temp) percentUsage=31%, usage=368688760, limit=104857600,
percentUsageMinDelta=1%;Parent:Usage(default:temp) percentUsage=351%, usage=368688760, limit=104857600,
percentUsageMinDelta=1%: Temp Store is Full (31% of 104857600). Stopping producer (ID:meme-34687-1370527272960-2:2:-1:1)
to prevent flooding queue://testqueue. See http://activemq.apache.org/producer-flow-control.html
for more info (blocking for: 1s) 
2013-06-06 15:01:52,141 [0.0.9.124:46281] INFO  Queue - Usage(default:temp:queue://testqueue:temp)
percentUsage=31%, usage=368688760, limit=104857600, percentUsageMinDelta=1%;Parent:Usage(default:temp)
percentUsage=351%, usage=368688760, limit=104857600, percentUsageMinDelta=1%: Temp Store is
Full (31% of 104857600). Stopping producer (ID:meme-34687-1370527272960-2:2:-1:1) to prevent
flooding queue://testqueue. See http://activemq.apache.org/producer-flow-control.html for
more info (blocking for: 1s) 

which shows the 351% usage, as per 100Mb temporary space. Now ActiveMQ is
fried.

I know that this config is broken, but wouldn't it make sense for the 20 Mb
limit to kick in? 

Otherwise, I have to assume that the aggregate memory usage of all the queues
has to be less than the memoryUsage limit?

thanks fr the reply, I'll be testing with other configurations and come back
with more questions on this thread :-)


-- 
JRF

Mime
View raw message