axis-java-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefano Bruna (JIRA)" <j...@apache.org>
Subject [jira] Created: (AXIS2-4749) Tribe's AtMostOnceInterceptor could lead to OutOfMemory under heavy load and big messages.
Date Tue, 22 Jun 2010 14:05:58 GMT
Tribe's AtMostOnceInterceptor could lead to OutOfMemory under heavy load and big messages.
------------------------------------------------------------------------------------------

                 Key: AXIS2-4749
                 URL: https://issues.apache.org/jira/browse/AXIS2-4749
             Project: Axis2
          Issue Type: Bug
    Affects Versions: 1.6
         Environment: Linux 64 bit, JDK 1.6, WSO2 Carbon 2.0.3
            Reporter: Stefano Bruna


Wa are stressing the WSO2 ESB with some cache mediator enabled. Messages are echanged by tribes
through the cluster's nodes. Under heavy load and with some big xml messages (1 mb per message)
the local variable Map<ChannelMessage, Long> receivedMessages is growing continuously
leading to a potential Out of Memory if the cleaning thread that runs every 5 minutes is not
fast enough to free up memory.

See: http://svn.apache.org/viewvc/axis/axis2/java/core/trunk/modules/clustering/src/org/apache/axis2/clustering/tribes/AtMostOnceInterceptor.java?view=markup

This is because when an object is passed to a HashMap as a key internally the original object
is kept. The Hasmap is using a private class Entry that has the hash, the orginal object used
to create the hash, and the value that, in this usage case, is a long.

Somewhere inside HashMap.java

        // private variables
	  final Object key;
        Object value;
        Entry next;
        final int hash;

	  // constructor 
        Entry(int i, Object obj, Object obj1, Entry entry)
        {
            value = obj1; <==== value, ok
            next = entry;
            key = obj; <======= orginal object !!!
            hash = i; <======== key, ok 
        }

So if the AtMostOnceInterceptor manages for example 50 msg/sec and a message is 1 mb we could
have within 5 minutes a memory usage for the messageReceived object of 6 GB.

A simple solution, if we dont'to accept all this as something by desing, could be to pass
to the HashMap the already calculated hash of the object (that is also the same method that
is called internally  int i = hash(obj.hashCode());) to not give the opportunity to the HashMap
to keep the actual object used to produce the key.

.....
// map
 private static final Map<Integer, Long> receivedMessagesHashCodes =
            new HashMap<Integer, Long>();

.....

Integer hashCode = new Integer(msg.hashCode());
            	
if (receivedMessagesHashCodes.get(hashCode) == null) {  // If it is a new message, keep track
of it
        receivedMessagesHashCodes.put(hashCode, System.currentTimeMillis());
        super.messageReceived(msg);
} else {  // If it is a duplicate message, discard it. i.e. dont call super.messageReceived
        log.info("Duplicate message received from " + TribesUtil.getName(msg.getAddress()));
}

etc...




-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@axis.apache.org
For additional commands, e-mail: java-dev-help@axis.apache.org


Mime
View raw message