activemq-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christopher L. Shannon (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMQ-6477) ReduceMemoryFootprint should apply to non-persistent messages and subscriptions
Date Mon, 24 Oct 2016 15:20:59 GMT

    [ https://issues.apache.org/jira/browse/AMQ-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15602290#comment-15602290
] 

Christopher L. Shannon commented on AMQ-6477:
---------------------------------------------

That's a good point about not sharing the message while doing the clearing which I think could
be an issue with topic subscriptions since the message will potentially be passed to multiple
subs.

When doing the clear in the subscriptions maybe we need to do something like add a synchronize
around the clear block:

{noformat}
if (isReduceMemoryFootprint()) {
  synchronized(node.getMessage()) {
    if (node.getMessage().isMarshalled()) {
      node.getMessage().clearUnMarshalledState();
    }
}
{noformat}

Ultimately, I agree about having the size count both states.  That would protect form OOM
errors for when the message contains both of the states while also trying to save memory when
it can.  The hard part is trying to estimate the size of the properties map.

> ReduceMemoryFootprint should apply to non-persistent messages and subscriptions
> -------------------------------------------------------------------------------
>
>                 Key: AMQ-6477
>                 URL: https://issues.apache.org/jira/browse/AMQ-6477
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.14.1
>            Reporter: Christopher L. Shannon
>            Assignee: Christopher L. Shannon
>
> There is a flag called reduceMemoryFootprint which will clear out the unmarshalled state
of a message after it is added to a queue/topic because that state isn't counted towards the
size and can lead to OOM errors.
> However, even setting this flag, I am still seeing some brokers run out of memory.  After
analyzing the heap dumps I have found 2 reasons for this:
> 1) Non-persistent messages do not have their unmarshalled state cleared.  This was done
because when a message is persisted it is guaranteed to have the marshalled state.  However
we can still clear the memory for non-persistent messages as long as we check to make sure
the marshalled state exists first, which it will for transports like TCP but won't exist for
the VM transport.
> 2) When a message is added to a subscription the properties are needed to check which
subscription messages can be dispatched to.  This causes the memory to be unmarshalled again.
 
> In the topic case, we should probably defer the clearing of the state until after the
message is added to a subscription because messages are immediately dispatched to topic subs
so we don't unnecessarily have to convert the data twice.  In the queue case it probably makes
sense to clear memory on add to the queue and on add to the subscription.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message