From commits-return-2522-archive-asf-public=cust-asf.ponee.io@bookkeeper.apache.org Sat Feb 3 01:23:07 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 2DD73180608 for ; Sat, 3 Feb 2018 01:23:07 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 1D5EE160C57; Sat, 3 Feb 2018 00:23:07 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 657BF160C49 for ; Sat, 3 Feb 2018 01:23:06 +0100 (CET) Received: (qmail 14014 invoked by uid 500); 3 Feb 2018 00:23:05 -0000 Mailing-List: contact commits-help@bookkeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: bookkeeper-dev@bookkeeper.apache.org Delivered-To: mailing list commits@bookkeeper.apache.org Received: (qmail 14005 invoked by uid 99); 3 Feb 2018 00:23:05 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 03 Feb 2018 00:23:05 +0000 Received: by gitbox.apache.org (ASF Mail Server at gitbox.apache.org, from userid 33) id A2EC185A0E; Sat, 3 Feb 2018 00:23:04 +0000 (UTC) Date: Sat, 03 Feb 2018 00:23:04 +0000 To: "commits@bookkeeper.apache.org" Subject: [bookkeeper] branch master updated: ISSUE #1063: Write keeps refCnt longer MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Message-ID: <151761738453.10291.2499996746545330011@gitbox.apache.org> From: sijie@apache.org X-Git-Host: gitbox.apache.org X-Git-Repo: bookkeeper X-Git-Refname: refs/heads/master X-Git-Reftype: branch X-Git-Oldrev: 49b011b3ba4d71482bb50be72f69b505c697aad4 X-Git-Newrev: 9d09a9c2a64b745271ef1c6dad9e5ab3ab3f2a5c X-Git-Rev: 9d09a9c2a64b745271ef1c6dad9e5ab3ab3f2a5c X-Git-NotificationType: ref_changed_plus_diff X-Git-Multimail-Version: 1.5.dev Auto-Submitted: auto-generated This is an automated email from the ASF dual-hosted git repository. sijie pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/bookkeeper.git The following commit(s) were added to refs/heads/master by this push: new 9d09a9c ISSUE #1063: Write keeps refCnt longer 9d09a9c is described below commit 9d09a9c2a64b745271ef1c6dad9e5ab3ab3f2a5c Author: JV Jujjuri AuthorDate: Fri Feb 2 16:22:56 2018 -0800 ISSUE #1063: Write keeps refCnt longer Descriptions of the changes in this PR: Current code keeps the toSend buffers until client receives resonses from all wq bookies. From the senders perspective, It is not required to keep the refcount on it at the PendingAddOp level, as the ref is taken at bookie client level. Keeping these buffers longer will increase the memory pressure on the client. Signed-off-by: Venkateswararao Jujjuri (JV) Master Issue: #1091 Author: JV Jujjuri Reviewers: Enrico Olivelli , Sijie Guo This closes #1091 from jvrao/bookkeeper-1063, closes #1063 --- .../org/apache/bookkeeper/client/PendingAddOp.java | 43 ++++++++++++++-------- 1 file changed, 27 insertions(+), 16 deletions(-) diff --git a/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingAddOp.java b/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingAddOp.java index 77f05c1..e386701 100644 --- a/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingAddOp.java +++ b/bookkeeper-server/src/main/java/org/apache/bookkeeper/client/PendingAddOp.java @@ -383,29 +383,40 @@ class PendingAddOp extends SafeRunnable implements WriteCallback { this.recyclerHandle = recyclerHandle; } + private void maybeRecycle() { - // The reference to PendingAddOp can be held in 3 places - // - LedgerHandle#pendingAddOp - // This reference is released when the callback is run - // - The executor - // Released after safeRun finishes - // - BookieClient - // Holds a reference from the point the addEntry requests are - // sent. - // The object can only be recycled after all references are - // released, otherwise we could end up recycling twice and all - // joy that goes along with that. - if (hasRun && callbackTriggered && pendingWriteRequests == 0) { - recycle(); + /** + * We have opportunity to recycle two objects here. + * PendingAddOp#toSend and LedgerHandle#pendingAddOp + * + * A. LedgerHandle#pendingAddOp: This can be released after all 3 conditions are met. + * - After the callback is run + * - After safeRun finished by the executor + * - Write responses are returned from all bookies + * as BookieClient Holds a reference from the point the addEntry requests are sent. + * + * B. ByteBuf can be released after 2 of the conditions are met. + * - After the callback is run as we will not retry the write after callback + * - After safeRun finished by the executor + * BookieClient takes and releases on this buffer immediately after sending the data. + * + * The object can only be recycled after the above conditions are met + * otherwise we could end up recycling twice and all + * joy that goes along with that. + */ + if (hasRun && callbackTriggered) { + ReferenceCountUtil.release(toSend); + toSend = null; + } + if (toSend == null && pendingWriteRequests == 0) { + recyclePendAddOpObject(); } } - private void recycle() { + private void recyclePendAddOpObject() { entryId = LedgerHandle.INVALID_ENTRY_ID; currentLedgerLength = -1; - ReferenceCountUtil.release(toSend); payload = null; - toSend = null; cb = null; ctx = null; ackSet.recycle(); -- To stop receiving notification emails like this one, please contact sijie@apache.org.