Mailing-List: contact dev-help@activemq.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@activemq.apache.org
Date: Tue, 30 Oct 2012 22:22:12 +0000 (UTC)
From: "Stirling Chow (JIRA)" <jira@apache.org>
To: dev@activemq.apache.org
Message-ID: <1395678373.47207.1351635732589.JavaMail.jiratomcat@arcas>
In-Reply-To: <1557548802.47192.1351635492329.JavaMail.jiratomcat@arcas>
Subject: [jira] [Updated] (AMQ-4147) Memory usage incorrectly updated across
 network of brokers when VMTransport is used.
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/AMQ-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Stirling Chow updated AMQ-4147:
-------------------------------

    Description: 
Symptom
=======
I was writing some unit tests based on {{org.apache.activemq.JmsMultipleBrokersTestSupport}}, and noticed the following issue:

When brokers are connected in a network using VMTransport (i.e., VMTransport on both the local and remote ends of a bridge), and a message moves from broker1 to broker2, the memory usage of the message continues to be accounted for on broker1 until the message is actually consumed on broker2.  

If the brokers are bridged using a non-VM transport (e.g., TCP), memory usage on broker1 is reduced as soon as the message is bridged to broker2, and broker2's memory usage increases until the message is consumed.

Cause
=====
The cause is the same as in AMQ-4116: {{Message.copy()}} copies the {{memoryUsage}} object.  As a result, when the message moves from broker1 to broker2, the original {{memoryUsage}} object, which refers to broker1 is copied through to broker2 and never updated to broker2's {{memoryUsage}} object.

Specifically, the message is copied from the local transport before being sent to the remote transport by this code:

{code:title=DemandForwardingBridgeSupport.java}
protected void serviceLocalCommand(Command command) {
    if (!disposed.get()) {
        try {
            if (command.isMessageDispatch()) {
                enqueueCounter.incrementAndGet();
                final MessageDispatch md = (MessageDispatch) command;
                final DemandSubscription sub = subscriptionMapByLocalId.get(md.getConsumerId());
                if (sub != null && md.getMessage() != null && sub.incrementOutstandingResponses()) {

                    if (suppressMessageDispatch(md, sub)) {
                        if (LOG.isDebugEnabled()) {
                            LOG.debug(configuration.getBrokerName() + " message not forwarded to " + remoteBrokerName + " because message came from there or fails networkTTL, brokerPath: " + Arrays.toString(md.getMessage().getBrokerPath()) + ", message: " + md.getMessage());
                        }
                        // still ack as it may be durable
                        try {
                            localBroker.oneway(new MessageAck(md, MessageAck.INDIVIDUAL_ACK_TYPE, 1));
                        } finally {
                            sub.decrementOutstandingResponses();
                        }
                        return;
                    }

                    Message message = configureMessage(md);
...

protected Message configureMessage(MessageDispatch md) {
    Message message = md.getMessage().copy();
...

{code}

This bug is not present when a non-VM transport is used for the bridge because {{Message.memoryUsage}} is marked transient *not* copied during serialization/deserialization over the transport.

Solution
========
Since this is at least the second bug to result from the current {{Message.copy()}} behaviour, I think "Solution 1" proposed from AMQ-4116 is starting to look attractive as both a solution to AMQ-4116 and this bug:

Modify {{Message.copy()}} so that it sets the resulting message's {{memoryUsage}} object to {{null}}.  This would make {{Message.copy()}} behave more similarly to serialization/deserialization.


  was:
Symptom
=======
I was writing some unit tests based on {{org.apache.activemq.JmsMultipleBrokersTestSupport}}, and noticed the following issue:

When brokers are connected in a network using VMTransport (i.e., VMTransport on both the local and remote ends of a bridge), and a message moves from broker1 to broker2, the memory usage of the message continues to be accounted for on broker1 until the message is actually consumed on broker2.  

If the brokers are bridged using a non-VM transport (e.g., TCP), memory usage on broker1 is reduced as soon as the message is bridged to broker2, and broker2's memory usage increases until the message is consumed.

Cause
=====
The cause is the same as in AMQ-4116: {{Message.copy()}} copies the {{memoryUsage}} object.  As a result, when the message moves from broker1 to broker2, the original {{memoryUsage}} object, which refers to broker1 is copied through to broker2 and never updated to broker2's {{memoryUsage}} object.

This bug is not present when a non-VM transport is used for the bridge because {{Message.memoryUsage}} is marked transient *not* copied during serialization/deserialization over the transport.

Solution
========
Since this is at least the second bug to result from the current {{Message.copy()}} behaviour, I think "Solution 1" proposed from AMQ-4116 is starting to look attractive as both a solution to AMQ-4116 and this bug:

Modify {{Message.copy()}} so that it sets the resulting message's {{memoryUsage}} object to {{null}}.  This would make {{Message.copy()}} behave more similarly to serialization/deserialization.


> Memory usage incorrectly updated across network of brokers when VMTransport is used.
> ------------------------------------------------------------------------------------
>
>                 Key: AMQ-4147
>                 URL: https://issues.apache.org/jira/browse/AMQ-4147
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.8.0
>            Reporter: Stirling Chow
>
> Symptom
> =======
> I was writing some unit tests based on {{org.apache.activemq.JmsMultipleBrokersTestSupport}}, and noticed the following issue:
> When brokers are connected in a network using VMTransport (i.e., VMTransport on both the local and remote ends of a bridge), and a message moves from broker1 to broker2, the memory usage of the message continues to be accounted for on broker1 until the message is actually consumed on broker2.  
> If the brokers are bridged using a non-VM transport (e.g., TCP), memory usage on broker1 is reduced as soon as the message is bridged to broker2, and broker2's memory usage increases until the message is consumed.
> Cause
> =====
> The cause is the same as in AMQ-4116: {{Message.copy()}} copies the {{memoryUsage}} object.  As a result, when the message moves from broker1 to broker2, the original {{memoryUsage}} object, which refers to broker1 is copied through to broker2 and never updated to broker2's {{memoryUsage}} object.
> Specifically, the message is copied from the local transport before being sent to the remote transport by this code:
> {code:title=DemandForwardingBridgeSupport.java}
> protected void serviceLocalCommand(Command command) {
>     if (!disposed.get()) {
>         try {
>             if (command.isMessageDispatch()) {
>                 enqueueCounter.incrementAndGet();
>                 final MessageDispatch md = (MessageDispatch) command;
>                 final DemandSubscription sub = subscriptionMapByLocalId.get(md.getConsumerId());
>                 if (sub != null && md.getMessage() != null && sub.incrementOutstandingResponses()) {
>                     if (suppressMessageDispatch(md, sub)) {
>                         if (LOG.isDebugEnabled()) {
>                             LOG.debug(configuration.getBrokerName() + " message not forwarded to " + remoteBrokerName + " because message came from there or fails networkTTL, brokerPath: " + Arrays.toString(md.getMessage().getBrokerPath()) + ", message: " + md.getMessage());
>                         }
>                         // still ack as it may be durable
>                         try {
>                             localBroker.oneway(new MessageAck(md, MessageAck.INDIVIDUAL_ACK_TYPE, 1));
>                         } finally {
>                             sub.decrementOutstandingResponses();
>                         }
>                         return;
>                     }
>                     Message message = configureMessage(md);
> ...
> protected Message configureMessage(MessageDispatch md) {
>     Message message = md.getMessage().copy();
> ...
> {code}
> This bug is not present when a non-VM transport is used for the bridge because {{Message.memoryUsage}} is marked transient *not* copied during serialization/deserialization over the transport.
> Solution
> ========
> Since this is at least the second bug to result from the current {{Message.copy()}} behaviour, I think "Solution 1" proposed from AMQ-4116 is starting to look attractive as both a solution to AMQ-4116 and this bug:
> Modify {{Message.copy()}} so that it sets the resulting message's {{memoryUsage}} object to {{null}}.  This would make {{Message.copy()}} behave more similarly to serialization/deserialization.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira