Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 98DE9200BA7 for ; Fri, 21 Oct 2016 14:52:06 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 9764F160AE8; Fri, 21 Oct 2016 12:52:06 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 8E8DD160AE0 for ; Fri, 21 Oct 2016 14:52:05 +0200 (CEST) Received: (qmail 33311 invoked by uid 500); 21 Oct 2016 12:52:04 -0000 Mailing-List: contact commits-help@qpid.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@qpid.apache.org Delivered-To: mailing list commits@qpid.apache.org Received: (qmail 33302 invoked by uid 99); 21 Oct 2016 12:52:04 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 21 Oct 2016 12:52:04 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id 53583C0EC8 for ; Fri, 21 Oct 2016 12:52:04 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.199 X-Spam-Level: X-Spam-Status: No, score=-1.199 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id OPXK45bfyrfr for ; Fri, 21 Oct 2016 12:52:02 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id CD7B95F1F5 for ; Fri, 21 Oct 2016 12:52:01 +0000 (UTC) Received: from svn01-us-west.apache.org (svn.apache.org [10.41.0.6]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 00EDBE017A for ; Fri, 21 Oct 2016 12:51:57 +0000 (UTC) Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id 138143A0046 for ; Fri, 21 Oct 2016 12:51:56 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1766014 - in /qpid/java/branches/6.0.x: ./ broker-plugins/amqp-0-10-protocol/src/main/java/org/apache/qpid/server/protocol/v0_10/ common/src/main/java/org/apache/qpid/bytebuffer/ Date: Fri, 21 Oct 2016 12:51:56 -0000 To: commits@qpid.apache.org From: lquack@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20161021125157.138143A0046@svn01-us-west.apache.org> archived-at: Fri, 21 Oct 2016 12:52:06 -0000 Author: lquack Date: Fri Oct 21 12:51:56 2016 New Revision: 1766014 URL: http://svn.apache.org/viewvc?rev=1766014&view=rev Log: QPID-7465: [Java Broker] Free memory in MessageMetaData_0_10 when it is no longer needed merged from trunk with: $ svn merge -c r1765828 ^/qpid/java/trunk minor merge conflicts manually resolved. Modified: qpid/java/branches/6.0.x/ (props changed) qpid/java/branches/6.0.x/broker-plugins/amqp-0-10-protocol/src/main/java/org/apache/qpid/server/protocol/v0_10/MessageMetaData_0_10.java qpid/java/branches/6.0.x/broker-plugins/amqp-0-10-protocol/src/main/java/org/apache/qpid/server/protocol/v0_10/ServerEncoder.java qpid/java/branches/6.0.x/common/src/main/java/org/apache/qpid/bytebuffer/QpidByteBuffer.java Propchange: qpid/java/branches/6.0.x/ ------------------------------------------------------------------------------ --- svn:mergeinfo (original) +++ svn:mergeinfo Fri Oct 21 12:51:56 2016 @@ -9,5 +9,5 @@ /qpid/branches/java-broker-vhost-refactor/java:1493674-1494547 /qpid/branches/java-network-refactor/qpid/java:805429-821809 /qpid/branches/qpid-2935/qpid/java:1061302-1072333 -/qpid/java/trunk:1715445-1715447,1715586,1715940,1716086-1716087,1716127-1716128,1716141,1716153,1716155,1716194,1716204,1716209,1716227,1716277,1716357,1716368,1716370,1716374,1716432,1716444-1716445,1716455,1716461,1716474,1716489,1716497,1716515,1716555,1716602,1716606-1716610,1716619,1716636,1717269,1717299,1717401,1717446,1717449,1717626,1717691,1717735,1717780,1718744,1718889,1718893,1718918,1718922,1719026,1719028,1719033,1719037,1719047,1719051,1720340,1720664,1721151,1721198,1722019-1722020,1722246,1722339,1722416,1722674,1722678,1722683,1722711,1723064,1723194,1723563,1724216,1724251,1724257,1724292,1724375,1724397,1724432,1724582,1724603,1724780,1724843-1724844,1725295,1725569,1725760,1726176,1726244-1726246,1726249,1726358,1726436,1726449,1726456,1726646,1726653,1726755,1726778,1727532,1727555,1727608,1727951,1727954,1728089,1728167,1728302,1728497,1728501,1728524,1728639,1728651,1728772,1729215,1729297,1729347,1729356,1729406,1729408,1729412,1729515,1729638,1729656-1729 657,1729783,1729828,1729832,1729841,1729851,1729886,1729904,1729973,1730019,1730025,1730052,1730072,1730088,1730494,1730499,1730547,1730559,1730567,1730578,1730585,1730651,1730697,1730712-1730713,1730805,1731029,1731110,1731210,1731225,1731444,1731551,1731612,1732184,1732452,1732461,1732465,1732525,1732812,1733467,1734452,1736478,1736751,1736838,1737804,1737835,1737853,1737984,1737992,1738119,1738135,1738231,1738271,1738607,1738610,1738731,1738914,1741702,1742257,1742284,1742339,1742544,1742900,1742926,1743161,1743228,1743383,1743982,1744012-1744013,1744046,1744123,1744157,1744276,1744403,1745424,1745450,1746140,1746273,1747526,1748254,1748723,1748818,1749349,1749399,1749482,1749524,1750359-1750360,1750943,1751433,1754251,1754354,1754392,1754429,1754510,1754550,1755561,1755957,1758628,1758640,1758766,1758964,1758980,1759774,1759783,1760032,1760337,1760522,1760546,1763653,1765350,1765609 +/qpid/java/trunk:1715445-1715447,1715586,1715940,1716086-1716087,1716127-1716128,1716141,1716153,1716155,1716194,1716204,1716209,1716227,1716277,1716357,1716368,1716370,1716374,1716432,1716444-1716445,1716455,1716461,1716474,1716489,1716497,1716515,1716555,1716602,1716606-1716610,1716619,1716636,1717269,1717299,1717401,1717446,1717449,1717626,1717691,1717735,1717780,1718744,1718889,1718893,1718918,1718922,1719026,1719028,1719033,1719037,1719047,1719051,1720340,1720664,1721151,1721198,1722019-1722020,1722246,1722339,1722416,1722674,1722678,1722683,1722711,1723064,1723194,1723563,1724216,1724251,1724257,1724292,1724375,1724397,1724432,1724582,1724603,1724780,1724843-1724844,1725295,1725569,1725760,1726176,1726244-1726246,1726249,1726358,1726436,1726449,1726456,1726646,1726653,1726755,1726778,1727532,1727555,1727608,1727951,1727954,1728089,1728167,1728302,1728497,1728501,1728524,1728639,1728651,1728772,1729215,1729297,1729347,1729356,1729406,1729408,1729412,1729515,1729638,1729656-1729 657,1729783,1729828,1729832,1729841,1729851,1729886,1729904,1729973,1730019,1730025,1730052,1730072,1730088,1730494,1730499,1730547,1730559,1730567,1730578,1730585,1730651,1730697,1730712-1730713,1730805,1731029,1731110,1731210,1731225,1731444,1731551,1731612,1732184,1732452,1732461,1732465,1732525,1732812,1733467,1734452,1736478,1736751,1736838,1737804,1737835,1737853,1737984,1737992,1738119,1738135,1738231,1738271,1738607,1738610,1738731,1738914,1741702,1742257,1742284,1742339,1742544,1742900,1742926,1743161,1743228,1743383,1743982,1744012-1744013,1744046,1744123,1744157,1744276,1744403,1745424,1745450,1746140,1746273,1747526,1748254,1748723,1748818,1749349,1749399,1749482,1749524,1750359-1750360,1750943,1751433,1754251,1754354,1754392,1754429,1754510,1754550,1755561,1755957,1758628,1758640,1758766,1758964,1758980,1759774,1759783,1760032,1760337,1760522,1760546,1763653,1765350,1765609,1765828 /qpid/trunk/qpid:796646-796653 Modified: qpid/java/branches/6.0.x/broker-plugins/amqp-0-10-protocol/src/main/java/org/apache/qpid/server/protocol/v0_10/MessageMetaData_0_10.java URL: http://svn.apache.org/viewvc/qpid/java/branches/6.0.x/broker-plugins/amqp-0-10-protocol/src/main/java/org/apache/qpid/server/protocol/v0_10/MessageMetaData_0_10.java?rev=1766014&r1=1766013&r2=1766014&view=diff ============================================================================== --- qpid/java/branches/6.0.x/broker-plugins/amqp-0-10-protocol/src/main/java/org/apache/qpid/server/protocol/v0_10/MessageMetaData_0_10.java (original) +++ qpid/java/branches/6.0.x/broker-plugins/amqp-0-10-protocol/src/main/java/org/apache/qpid/server/protocol/v0_10/MessageMetaData_0_10.java Fri Oct 21 12:51:56 2016 @@ -20,6 +20,11 @@ */ package org.apache.qpid.server.protocol.v0_10; +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.List; + import org.apache.qpid.bytebuffer.QpidByteBuffer; import org.apache.qpid.server.message.AMQMessageHeader; import org.apache.qpid.server.plugin.MessageMetaDataType; @@ -30,13 +35,6 @@ import org.apache.qpid.transport.Message import org.apache.qpid.transport.MessageProperties; import org.apache.qpid.transport.MessageTransfer; import org.apache.qpid.transport.Struct; -import org.apache.qpid.transport.codec.BBEncoder; - -import java.nio.ByteBuffer; -import java.util.ArrayList; -import java.util.Collection; -import java.util.Collections; -import java.util.List; public class MessageMetaData_0_10 implements StorableMessageMetaData { @@ -86,7 +84,7 @@ public class MessageMetaData_0_10 implem return TYPE; } - public int getStorableSize() + public synchronized int getStorableSize() { QpidByteBuffer buf = _encoded; @@ -96,13 +94,12 @@ public class MessageMetaData_0_10 implem _encoded = buf; } - //TODO -- need to add stuff return buf.limit(); } private QpidByteBuffer encodeAsBuffer() { - ServerEncoder encoder = new ServerEncoder(ENCODER_SIZE); + ServerEncoder encoder = new ServerEncoder(ENCODER_SIZE, false); encoder.writeInt64(_arrivalTime); encoder.writeInt32(_bodySize); @@ -144,32 +141,22 @@ public class MessageMetaData_0_10 implem return buf; } - public int writeToBuffer(QpidByteBuffer dest) + public synchronized int writeToBuffer(QpidByteBuffer dest) { - QpidByteBuffer buf = _encoded; - - if(buf == null) + if (_encoded == null) { - buf = encodeAsBuffer(); - _encoded = buf; + _encoded = encodeAsBuffer(); } - - buf = buf.duplicate(); - - buf.position(0); - - if(dest.remaining() < buf.limit()) - { - buf.limit(dest.remaining()); - } - dest.put(buf); - final int length = buf.limit(); - buf.dispose(); - return length; + dest.put(_encoded); + final int bytesWritten = _encoded.limit(); + // We have special knowledge that we no longer need the encoded form after this call + // to reduce memory usage associated with the metadata free the encoded form here (QPID-7465) + clearEncodedForm(); + return bytesWritten; } @Override - public Collection asByteBuffers() + public synchronized Collection asByteBuffers() { QpidByteBuffer buf = _encoded; @@ -199,13 +186,17 @@ public class MessageMetaData_0_10 implem @Override public void dispose() { - + clearEncodedForm(); } @Override - public void clearEncodedForm() + public synchronized void clearEncodedForm() { - + if (_encoded != null) + { + _encoded.dispose(); + _encoded = null; + } } public String getRoutingKey() Modified: qpid/java/branches/6.0.x/broker-plugins/amqp-0-10-protocol/src/main/java/org/apache/qpid/server/protocol/v0_10/ServerEncoder.java URL: http://svn.apache.org/viewvc/qpid/java/branches/6.0.x/broker-plugins/amqp-0-10-protocol/src/main/java/org/apache/qpid/server/protocol/v0_10/ServerEncoder.java?rev=1766014&r1=1766013&r2=1766014&view=diff ============================================================================== --- qpid/java/branches/6.0.x/broker-plugins/amqp-0-10-protocol/src/main/java/org/apache/qpid/server/protocol/v0_10/ServerEncoder.java (original) +++ qpid/java/branches/6.0.x/broker-plugins/amqp-0-10-protocol/src/main/java/org/apache/qpid/server/protocol/v0_10/ServerEncoder.java Fri Oct 21 12:51:56 2016 @@ -30,21 +30,23 @@ import org.apache.qpid.transport.codec.A public final class ServerEncoder extends AbstractEncoder { - public static final int DEFAULT_CAPACITY = 8192; + public static final int DEFAULT_CAPACITY = 256 * 1024; + private final boolean _useDirectMemory; private final int _threshold; private QpidByteBuffer _out; private int _initialCapacity; public ServerEncoder() { - this(DEFAULT_CAPACITY); + this(QpidByteBuffer.getPooledBufferSize(), true); } - public ServerEncoder(int capacity) + public ServerEncoder(int capacity, boolean useDirectMemory) { - _initialCapacity = capacity; - _threshold = capacity/16; - _out = QpidByteBuffer.allocateDirect(capacity); + _useDirectMemory = useDirectMemory; + _initialCapacity = (capacity > 0 ? capacity : DEFAULT_CAPACITY); + _threshold = Math.min(_initialCapacity/16, 256); + _out = QpidByteBuffer.allocate(useDirectMemory, _initialCapacity); } public void init() @@ -52,7 +54,7 @@ public final class ServerEncoder extends if(_out.capacity() < _threshold) { _out.dispose(); - _out = QpidByteBuffer.allocateDirect(_initialCapacity); + _out = QpidByteBuffer.allocate(_useDirectMemory, _initialCapacity); } else { @@ -81,7 +83,7 @@ public final class ServerEncoder extends { QpidByteBuffer old = _out; int capacity = old.capacity(); - _out = QpidByteBuffer.allocateDirect(Math.max(Math.max(capacity + size, 2*capacity), _initialCapacity)); + _out = QpidByteBuffer.allocate(_useDirectMemory, Math.max(Math.max(capacity + size, 2 * capacity), _initialCapacity)); old.flip(); _out.put(old); old.dispose(); Modified: qpid/java/branches/6.0.x/common/src/main/java/org/apache/qpid/bytebuffer/QpidByteBuffer.java URL: http://svn.apache.org/viewvc/qpid/java/branches/6.0.x/common/src/main/java/org/apache/qpid/bytebuffer/QpidByteBuffer.java?rev=1766014&r1=1766013&r2=1766014&view=diff ============================================================================== --- qpid/java/branches/6.0.x/common/src/main/java/org/apache/qpid/bytebuffer/QpidByteBuffer.java (original) +++ qpid/java/branches/6.0.x/common/src/main/java/org/apache/qpid/bytebuffer/QpidByteBuffer.java Fri Oct 21 12:51:56 2016 @@ -701,6 +701,11 @@ public final class QpidByteBuffer _isPoolInitialized = true; } + public static int getPooledBufferSize() + { + return _pooledBufferSize; + } + private final class BufferInputStream extends InputStream { --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscribe@qpid.apache.org For additional commands, e-mail: commits-help@qpid.apache.org