activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stirling Chow (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (AMQ-4147) Memory usage incorrectly updated across network of brokers when VMTransport is used.
Date Tue, 30 Oct 2012 22:22:12 GMT

     [ https://issues.apache.org/jira/browse/AMQ-4147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Stirling Chow updated AMQ-4147:
-------------------------------

    Description: 
Symptom
=======
I was writing some unit tests based on {{org.apache.activemq.JmsMultipleBrokersTestSupport}},
and noticed the following issue:

When brokers are connected in a network using VMTransport (i.e., VMTransport on both the local
and remote ends of a bridge), and a message moves from broker1 to broker2, the memory usage
of the message continues to be accounted for on broker1 until the message is actually consumed
on broker2.  

If the brokers are bridged using a non-VM transport (e.g., TCP), memory usage on broker1 is
reduced as soon as the message is bridged to broker2, and broker2's memory usage increases
until the message is consumed.

Cause
=====
The cause is the same as in AMQ-4116: {{Message.copy()}} copies the {{memoryUsage}} object.
 As a result, when the message moves from broker1 to broker2, the original {{memoryUsage}}
object, which refers to broker1 is copied through to broker2 and never updated to broker2's
{{memoryUsage}} object.

Specifically, the message is copied from the local transport before being sent to the remote
transport by this code:

{code:title=DemandForwardingBridgeSupport.java}
protected void serviceLocalCommand(Command command) {
    if (!disposed.get()) {
        try {
            if (command.isMessageDispatch()) {
                enqueueCounter.incrementAndGet();
                final MessageDispatch md = (MessageDispatch) command;
                final DemandSubscription sub = subscriptionMapByLocalId.get(md.getConsumerId());
                if (sub != null && md.getMessage() != null && sub.incrementOutstandingResponses())
{

                    if (suppressMessageDispatch(md, sub)) {
                        if (LOG.isDebugEnabled()) {
                            LOG.debug(configuration.getBrokerName() + " message not forwarded
to " + remoteBrokerName + " because message came from there or fails networkTTL, brokerPath:
" + Arrays.toString(md.getMessage().getBrokerPath()) + ", message: " + md.getMessage());
                        }
                        // still ack as it may be durable
                        try {
                            localBroker.oneway(new MessageAck(md, MessageAck.INDIVIDUAL_ACK_TYPE,
1));
                        } finally {
                            sub.decrementOutstandingResponses();
                        }
                        return;
                    }

                    Message message = configureMessage(md);
...

protected Message configureMessage(MessageDispatch md) {
    Message message = md.getMessage().copy();
...

{code}

This bug is not present when a non-VM transport is used for the bridge because {{Message.memoryUsage}}
is marked transient *not* copied during serialization/deserialization over the transport.

Solution
========
Since this is at least the second bug to result from the current {{Message.copy()}} behaviour,
I think "Solution 1" proposed from AMQ-4116 is starting to look attractive as both a solution
to AMQ-4116 and this bug:

Modify {{Message.copy()}} so that it sets the resulting message's {{memoryUsage}} object to
{{null}}.  This would make {{Message.copy()}} behave more similarly to serialization/deserialization.



  was:
Symptom
=======
I was writing some unit tests based on {{org.apache.activemq.JmsMultipleBrokersTestSupport}},
and noticed the following issue:

When brokers are connected in a network using VMTransport (i.e., VMTransport on both the local
and remote ends of a bridge), and a message moves from broker1 to broker2, the memory usage
of the message continues to be accounted for on broker1 until the message is actually consumed
on broker2.  

If the brokers are bridged using a non-VM transport (e.g., TCP), memory usage on broker1 is
reduced as soon as the message is bridged to broker2, and broker2's memory usage increases
until the message is consumed.

Cause
=====
The cause is the same as in AMQ-4116: {{Message.copy()}} copies the {{memoryUsage}} object.
 As a result, when the message moves from broker1 to broker2, the original {{memoryUsage}}
object, which refers to broker1 is copied through to broker2 and never updated to broker2's
{{memoryUsage}} object.

This bug is not present when a non-VM transport is used for the bridge because {{Message.memoryUsage}}
is marked transient *not* copied during serialization/deserialization over the transport.

Solution
========
Since this is at least the second bug to result from the current {{Message.copy()}} behaviour,
I think "Solution 1" proposed from AMQ-4116 is starting to look attractive as both a solution
to AMQ-4116 and this bug:

Modify {{Message.copy()}} so that it sets the resulting message's {{memoryUsage}} object to
{{null}}.  This would make {{Message.copy()}} behave more similarly to serialization/deserialization.



    
> Memory usage incorrectly updated across network of brokers when VMTransport is used.
> ------------------------------------------------------------------------------------
>
>                 Key: AMQ-4147
>                 URL: https://issues.apache.org/jira/browse/AMQ-4147
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.8.0
>            Reporter: Stirling Chow
>
> Symptom
> =======
> I was writing some unit tests based on {{org.apache.activemq.JmsMultipleBrokersTestSupport}},
and noticed the following issue:
> When brokers are connected in a network using VMTransport (i.e., VMTransport on both
the local and remote ends of a bridge), and a message moves from broker1 to broker2, the memory
usage of the message continues to be accounted for on broker1 until the message is actually
consumed on broker2.  
> If the brokers are bridged using a non-VM transport (e.g., TCP), memory usage on broker1
is reduced as soon as the message is bridged to broker2, and broker2's memory usage increases
until the message is consumed.
> Cause
> =====
> The cause is the same as in AMQ-4116: {{Message.copy()}} copies the {{memoryUsage}} object.
 As a result, when the message moves from broker1 to broker2, the original {{memoryUsage}}
object, which refers to broker1 is copied through to broker2 and never updated to broker2's
{{memoryUsage}} object.
> Specifically, the message is copied from the local transport before being sent to the
remote transport by this code:
> {code:title=DemandForwardingBridgeSupport.java}
> protected void serviceLocalCommand(Command command) {
>     if (!disposed.get()) {
>         try {
>             if (command.isMessageDispatch()) {
>                 enqueueCounter.incrementAndGet();
>                 final MessageDispatch md = (MessageDispatch) command;
>                 final DemandSubscription sub = subscriptionMapByLocalId.get(md.getConsumerId());
>                 if (sub != null && md.getMessage() != null && sub.incrementOutstandingResponses())
{
>                     if (suppressMessageDispatch(md, sub)) {
>                         if (LOG.isDebugEnabled()) {
>                             LOG.debug(configuration.getBrokerName() + " message not forwarded
to " + remoteBrokerName + " because message came from there or fails networkTTL, brokerPath:
" + Arrays.toString(md.getMessage().getBrokerPath()) + ", message: " + md.getMessage());
>                         }
>                         // still ack as it may be durable
>                         try {
>                             localBroker.oneway(new MessageAck(md, MessageAck.INDIVIDUAL_ACK_TYPE,
1));
>                         } finally {
>                             sub.decrementOutstandingResponses();
>                         }
>                         return;
>                     }
>                     Message message = configureMessage(md);
> ...
> protected Message configureMessage(MessageDispatch md) {
>     Message message = md.getMessage().copy();
> ...
> {code}
> This bug is not present when a non-VM transport is used for the bridge because {{Message.memoryUsage}}
is marked transient *not* copied during serialization/deserialization over the transport.
> Solution
> ========
> Since this is at least the second bug to result from the current {{Message.copy()}} behaviour,
I think "Solution 1" proposed from AMQ-4116 is starting to look attractive as both a solution
to AMQ-4116 and this bug:
> Modify {{Message.copy()}} so that it sets the resulting message's {{memoryUsage}} object
to {{null}}.  This would make {{Message.copy()}} behave more similarly to serialization/deserialization.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message