activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cybexion <cybex...@email.de>
Subject Re: Performance in relation to queue size
Date Mon, 26 Jan 2009 16:36:38 GMT

Hi,

I tried all this without success.
The more messages are in the queue the slower it gets. My system was running
during the whole weekend now. The first hour I had about 900 messages per
second (JMS sending took about 0ms). Then after about 1-2 hours JMS sending
suddently takes 1 ore more seconds.

My config looks like this (just the interesting stuff):
        <bean id="store"
class="org.apache.activemq.store.amq.AMQPersistenceAdapter">
                <property name="directory" value="${activemq.base}/data"/>
                <!-- 1GB = 1073741824 -->
                <property name="maxFileLength"          value="1073741824"/>
                <property name="maxReferenceFileLength" value="104857600"/>
                <property name="checkpointInterval"     value="60000"/>
                <property name="cleanupInterval"        value="30000"/>
                <property name="persistentIndex"        value="true"/>
                <property name="indexMaxBinSize"        value="10485760"/>
                <property name="indexBinSize"           value="1048576"/>
                <property name="indexPageSize"          value="65536"/>
        </bean>

  <broker xmlns="http://activemq.apache.org/schema/core"
        persistent="true"
        advisorySupport="false"
        deleteAllMessagesOnStartup="false"
        useJmx="true"
        monitorConnectionSplits="false"
        splitSystemUsageForProducersConsumers="false"
        brokerName="localhost"
        dataDirectory="${activemq.base}/data"
        persistenceAdapter="#store"
        >


       <!--  The maximum about of space the broker will use before slowing
down producers -->
        <systemUsage>
            <systemUsage sendFailIfNoSpace="true">
                <memoryUsage>
                    <memoryUsage limit="1gb"/>
                </memoryUsage>
                <storeUsage>
                    <storeUsage limit="30gb" store="#store"/>
                </storeUsage>

                <!-- temp usage is used with non persistent messages only
-->
                <tempUsage>
                    <tempUsage limit="64mb"/>
                </tempUsage>
            </systemUsage>
        </systemUsage>

Anyone else having an idea?
Next thing I try is to limit the queue size to lets say on GB and see if
performance is fine if the size is limited.






Gary Tully wrote:
> 
>> Well, the main question is now:
>> 1. Why is the queue getting slower when there are a lot of messages
>> inside?
> So with X million messages, enqueue times are 4-7 seconds?
> This is probabally because the hashIndex is limited, I think you need
> to make the indexMaxBinSize quite large and use the same value for
> indexBinSize so that no resizing takes place at runtime.  This should
> speed up index insertion.
> Also increasing the indexPageSize may help by increasing the quantity
> of data persisted in one write.
> 
> After that, this may be indicative of a problem with the single large
> hash index file.
> 
> 
> 
>> 2. How can I change the size of the data-queue-data-959 files?
> The size of the files in the reference store are controlled by the
> maxReferenceFileLength attribute.
> 
>> Would it make sense to have a few big of those files instead of a lot of
>> small one?
> Not sure, I think this is only relevant to cleanup, a large file with
> lots of references may take a long time to be unferenced and ready for
> cleanup.
> 
>> 3. What is this file for: hash-index-queue-data_queue#3a#2f#2ftracking
> This is the persistent part of the persistent hashIndex
> 
> The following link gives a overview of the amq persistence adapter
> explaining the layout etc:
> http://activemq.apache.org/amq-message-store.html
> 
> 2009/1/23 Cybexion <cybexion@email.de>:
>>
>> Hello,
>>
>> I changed my AMQPersistenceAdapter to write 1 GB files (total store size
>> is
>> 30GB)
>>        <bean id="store"
>> class="org.apache.activemq.store.amq.AMQPersistenceAdapter">
>>                <property name="directory" value="${activemq.base}/data"/>
>>                <!-- 1GB = 1073741824 -->
>>                <property name="maxFileLength" value="1073741824"/>
>>                <property name="checkpointInterval" value="60000"/>
>>                <property name="cleanupInterval" value="30000"/>
>>                <property name="persistentIndex" value="true"/>
>>        </bean>
>>
>> After that I created messages during the whole night into the queue.
>> The result looks like this:
>>
>> ls -l data/journal
>> -rw-r--r-- 1 activemq java 1073741777 2009-01-22 21:06 data-2
>> -rw-r--r-- 1 activemq java 1073741637 2009-01-22 23:17 data-3
>> -rw-r--r-- 1 activemq java 1073741573 2009-01-23 01:17 data-4
>> -rw-r--r-- 1 activemq java 1073741815 2009-01-23 03:40 data-5
>> -rw-r--r-- 1 activemq java 1073741674 2009-01-23 06:48 data-6
>> -rw-r--r-- 1 activemq java 1073217536 2009-01-23 11:22 data-7
>> -rw-r--r-- 1 activemq java       2088 2009-01-23 11:22 data-control
>>
>> About 5.2 GB of data has been created.
>>
>> These are the hundreds of files containing the keys to the messages.
>> ls -l data/kr-store/data
>> -rw-r--r-- 1 activemq java    2097142 2009-01-23 11:04
>> data-queue-data-957
>> -rw-r--r-- 1 activemq java    2097117 2009-01-23 11:11
>> data-queue-data-958
>> -rw-r--r-- 1 activemq java    2068877 2009-01-23 11:21
>> data-queue-data-959
>> -rw-r--r-- 1 activemq java 1462429697 2009-01-23 11:24
>> hash-index-queue-data_queue#3a#2f#2ftracking
>> -rw-r--r-- 1 activemq java        153 2009-01-22 17:10
>> index-container-roots
>> -rw-r--r-- 1 activemq java  705609480 2009-01-23 11:24 index-queue-data
>> -rw-r--r-- 1 activemq java          0 2009-01-22 17:09 lock
>>
>> ls -l | wc
>>    721    5762   54026
>>
>> Here are some additional JMX infos about the broker
>>
>> ActiveMQ Broker Memory percent Usage    :       0,00
>> ActiveMQ Broker Store percent Usage     :       26,00
>> ActiveMQ Broker Temp percent Usage      :       0,00
>> ActiveMQ Broker Consumer Count  :       1,00
>> ActiveMQ Broker Total Dequeue Count     :       3.629.262,00
>> ActiveMQ Broker Total Enqueue Count     :       16.132.693,00
>> ActiveMQ Broker Total Message Count     :       12.503.432,00
>>
>> Now to the bad news:
>> It looks like the more messages are in the queue, the slower it gets to
>> send
>> new messages into it (using JmsTemplate.send() method).
>> When I currently run I java programm that inserts messages into the Queue
>> I
>> have response times of 4-7 seconds !!!!
>> This is not good.
>> Here is an extract of the AMQ log file:
>> 2009-01-22 18:08:11,229 [ueue://tracking] INFO
>> kaha.impl.index.hash.HashIndex - Resize hash bins
>> queue-data_queue#3a#2f#2ftracking from 1024 to 2048
>> 2009-01-22 18:10:24,642 [ueue://tracking] INFO
>> kaha.impl.index.hash.HashIndex - Resize hash bins
>> queue-data_queue#3a#2f#2ftracking from 2048 to 4096
>> 2009-01-22 18:13:35,224 [ueue://tracking] INFO
>> kaha.impl.index.hash.HashIndex - Resize hash bins
>> queue-data_queue#3a#2f#2ftracking from 4096 to 8192
>> 2009-01-22 18:18:12,564 [ueue://tracking] INFO
>> kaha.impl.index.hash.HashIndex - Resize hash bins
>> queue-data_queue#3a#2f#2ftracking from 8192 to 16384
>>
>> Well, the main question is now:
>> 1. Why is the queue getting slower when there are a lot of messages
>> inside?
>> 2. How can I change the size of the data-queue-data-959 files?
>> Would it make sense to have a few big of those files instead of a lot of
>> small one?
>> 3. What is this file for: hash-index-queue-data_queue#3a#2f#2ftracking
>>
>> Thanks and regards
>>
>> Jörg
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Performance-in-relation-to-queue-size-tp21601585p21622121.html
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
> 
> 
> 
> -- 
> http://blog.garytully.com
> 
> Open Source SOA
> http://FUSESource.com
> 
> 

-- 
View this message in context: http://www.nabble.com/Performance-in-relation-to-queue-size-tp21601585p21668680.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message