activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Taras Tielkes (JIRA)" <j...@apache.org>
Subject [jira] Updated: (AMQ-2660) OutOfMemoryError trying to consume big message
Date Tue, 23 Mar 2010 22:33:46 GMT

     [ https://issues.apache.org/activemq/browse/AMQ-2660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Taras Tielkes updated AMQ-2660:
-------------------------------

    Description: 
Running 5.3.1 broken on Win64, using Sun 1.6 x64 JVM.
Maximum heap size is set to 1024M.

We're sending and receiving huge messages over ActiveMQ, and cannot get the system running
in a stable way.
For purposes of testing, we're sending text messages of 100M characters, I guess this means
200MB in-memory, since Strings are stored as UTF16 in JVM memory.
However, with the broker having a 1GB heap this should work - we assume.

Testing with a single producer, single consumer.

We're using a slightly modified activemq.xml:
1) "systemUsage" block enabled, and configured as shown below:
-------
        <systemUsage>
            <systemUsage>
                <memoryUsage>
                    <memoryUsage limit="500 mb"/>
                </memoryUsage>
                <storeUsage>
                    <storeUsage limit="1 gb" name="foo"/>
                </storeUsage>
                <tempUsage>
                    <tempUsage limit="100 mb"/>
                </tempUsage>
            </systemUsage>
        </systemUsage>
-------
2) producerFlowControl disabled, as shown below:
-------
<policyEntry queue=">" producerFlowControl="false" memoryLimit="1mb">
-------

Description of symptoms:
1) Starting up producer, let it run until store gets full. This typically works out to 9 or
10 messages, each being 100M characters in size. From this I'm inferring that ActiveMQ stores
these in UTF-8 format.
2) Starting up consumer

When starting the consumer, ActiveMQ broker fails with OOM shown below:
-------
ERROR | Failed to page in more queue messages
java.lang.OutOfMemoryError: Java heap space
        at org.apache.activemq.protobuf.BaseMessage.mergeFramed(BaseMessage.java:228)
        at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:681)
        at org.apache.activemq.store.kahadb.KahaDBStore.loadMessage(KahaDBStore.java:550)
        at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore$5.execute(KahaDBStore.java:242)
        at org.apache.kahadb.page.Transaction.execute(Transaction.java:728)
        at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.recoverNextMessages(KahaDBStore.java:235)
        at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:97)
        at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:251)
        at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.hasNext(AbstractStoreCursor.java:142)
        at org.apache.activemq.broker.region.cursors.StoreQueueCursor.hasNext(StoreQueueCursor.java:131)
        at org.apache.activemq.broker.region.Queue.doPageIn(Queue.java:1447)
        at org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:1585)
        at org.apache.activemq.broker.region.Queue.iterate(Queue.java:1219)
        at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:98)
        at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:36)
-------
Consumer keeps waiting, and never actually succeeds in consuming a message.


  was:
Running 5.3.1 broken on Win64, using Sun 1.6 x64 JVM.
Maximum heap size is set to 1024M.

We're sending and receiving huge messages over ActiveMQ, and cannot get the system running
in a stable way.
For purposes of testing, we're sending text messages of 100M characters, I guess this means
200MB in-memory, since String are stored as UTF16 in JVM memory.

Testing with a single producer, single consumer.

We're using a slightly modified activemq.xml:
1) "systemUsage" block enabled, and configured as shown below:
-------
        <systemUsage>
            <systemUsage>
                <memoryUsage>
                    <memoryUsage limit="500 mb"/>
                </memoryUsage>
                <storeUsage>
                    <storeUsage limit="1 gb" name="foo"/>
                </storeUsage>
                <tempUsage>
                    <tempUsage limit="100 mb"/>
                </tempUsage>
            </systemUsage>
        </systemUsage>
-------
2) producerFlowControl disabled, as shown below:
-------
<policyEntry queue=">" producerFlowControl="false" memoryLimit="1mb">
-------

Description of symptoms:
1) Starting up producer, let it run until store gets full. This typically works out to 9 or
10 messages, each being 100M characters in size. From this I'm inferring that ActiveMQ stores
these in UTF-8 format.
2) Starting up consumer

When starting the consumer, ActiveMQ broker fails with OOM shown below:
-------
ERROR | Failed to page in more queue messages
java.lang.OutOfMemoryError: Java heap space
        at org.apache.activemq.protobuf.BaseMessage.mergeFramed(BaseMessage.java:228)
        at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:681)
        at org.apache.activemq.store.kahadb.KahaDBStore.loadMessage(KahaDBStore.java:550)
        at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore$5.execute(KahaDBStore.java:242)
        at org.apache.kahadb.page.Transaction.execute(Transaction.java:728)
        at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.recoverNextMessages(KahaDBStore.java:235)
        at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:97)
        at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:251)
        at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.hasNext(AbstractStoreCursor.java:142)
        at org.apache.activemq.broker.region.cursors.StoreQueueCursor.hasNext(StoreQueueCursor.java:131)
        at org.apache.activemq.broker.region.Queue.doPageIn(Queue.java:1447)
        at org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:1585)
        at org.apache.activemq.broker.region.Queue.iterate(Queue.java:1219)
        at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:98)
        at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:36)
-------
Consumer keeps waiting, and never actually succeeds in consuming a message.



> OutOfMemoryError trying to consume big message
> ----------------------------------------------
>
>                 Key: AMQ-2660
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2660
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.3.1
>            Reporter: Taras Tielkes
>
> Running 5.3.1 broken on Win64, using Sun 1.6 x64 JVM.
> Maximum heap size is set to 1024M.
> We're sending and receiving huge messages over ActiveMQ, and cannot get the system running
in a stable way.
> For purposes of testing, we're sending text messages of 100M characters, I guess this
means 200MB in-memory, since Strings are stored as UTF16 in JVM memory.
> However, with the broker having a 1GB heap this should work - we assume.
> Testing with a single producer, single consumer.
> We're using a slightly modified activemq.xml:
> 1) "systemUsage" block enabled, and configured as shown below:
> -------
>         <systemUsage>
>             <systemUsage>
>                 <memoryUsage>
>                     <memoryUsage limit="500 mb"/>
>                 </memoryUsage>
>                 <storeUsage>
>                     <storeUsage limit="1 gb" name="foo"/>
>                 </storeUsage>
>                 <tempUsage>
>                     <tempUsage limit="100 mb"/>
>                 </tempUsage>
>             </systemUsage>
>         </systemUsage>
> -------
> 2) producerFlowControl disabled, as shown below:
> -------
> <policyEntry queue=">" producerFlowControl="false" memoryLimit="1mb">
> -------
> Description of symptoms:
> 1) Starting up producer, let it run until store gets full. This typically works out to
9 or 10 messages, each being 100M characters in size. From this I'm inferring that ActiveMQ
stores these in UTF-8 format.
> 2) Starting up consumer
> When starting the consumer, ActiveMQ broker fails with OOM shown below:
> -------
> ERROR | Failed to page in more queue messages
> java.lang.OutOfMemoryError: Java heap space
>         at org.apache.activemq.protobuf.BaseMessage.mergeFramed(BaseMessage.java:228)
>         at org.apache.activemq.store.kahadb.MessageDatabase.load(MessageDatabase.java:681)
>         at org.apache.activemq.store.kahadb.KahaDBStore.loadMessage(KahaDBStore.java:550)
>         at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore$5.execute(KahaDBStore.java:242)
>         at org.apache.kahadb.page.Transaction.execute(Transaction.java:728)
>         at org.apache.activemq.store.kahadb.KahaDBStore$KahaDBMessageStore.recoverNextMessages(KahaDBStore.java:235)
>         at org.apache.activemq.broker.region.cursors.QueueStorePrefetch.doFillBatch(QueueStorePrefetch.java:97)
>         at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.fillBatch(AbstractStoreCursor.java:251)
>         at org.apache.activemq.broker.region.cursors.AbstractStoreCursor.hasNext(AbstractStoreCursor.java:142)
>         at org.apache.activemq.broker.region.cursors.StoreQueueCursor.hasNext(StoreQueueCursor.java:131)
>         at org.apache.activemq.broker.region.Queue.doPageIn(Queue.java:1447)
>         at org.apache.activemq.broker.region.Queue.pageInMessages(Queue.java:1585)
>         at org.apache.activemq.broker.region.Queue.iterate(Queue.java:1219)
>         at org.apache.activemq.thread.DedicatedTaskRunner.runTask(DedicatedTaskRunner.java:98)
>         at org.apache.activemq.thread.DedicatedTaskRunner$1.run(DedicatedTaskRunner.java:36)
> -------
> Consumer keeps waiting, and never actually succeeds in consuming a message.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message