Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 23FBF200AE4 for ; Fri, 24 Jun 2016 19:02:24 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id 20ECA160A58; Fri, 24 Jun 2016 17:02:24 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 42CB4160A2E for ; Fri, 24 Jun 2016 19:02:23 +0200 (CEST) Received: (qmail 89166 invoked by uid 500); 24 Jun 2016 17:02:22 -0000 Mailing-List: contact notifications-help@asterixdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@asterixdb.apache.org Delivered-To: mailing list notifications@asterixdb.apache.org Received: (qmail 89157 invoked by uid 99); 24 Jun 2016 17:02:22 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 24 Jun 2016 17:02:22 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 1143B180F27 for ; Fri, 24 Jun 2016 17:02:22 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.126 X-Spam-Level: ** X-Spam-Status: No, score=2.126 tagged_above=-999 required=6.31 tests=[MISSING_HEADERS=1.207, SPF_FAIL=0.919] autolearn=disabled Received: from mx2-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id xXto2-xUsPeR for ; Fri, 24 Jun 2016 17:02:19 +0000 (UTC) Received: from unhygienix.ics.uci.edu (unhygienix.ics.uci.edu [128.195.14.130]) by mx2-lw-eu.apache.org (ASF Mail Server at mx2-lw-eu.apache.org) with ESMTP id E1D5F5F36A for ; Fri, 24 Jun 2016 17:02:18 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by unhygienix.ics.uci.edu (Postfix) with ESMTP id EEF75241E75; Fri, 24 Jun 2016 10:02:17 -0700 (PDT) Date: Fri, 24 Jun 2016 10:02:17 -0700 From: "Michael Blow (Code Review)" CC: Jenkins , abdullah alamoudi Reply-To: michael.blow@couchbase.com X-Gerrit-MessageType: merged Subject: Change in asterixdb[master]: Cleanup Large Page Logic X-Gerrit-Change-Id: I26f9a8e4530edfda1b5f78114a5c8038bf8a27f4 X-Gerrit-ChangeURL: X-Gerrit-Commit: dc58d30d75a76fcc52f60febc5a8a94956230553 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.8.4 Message-Id: <20160624170217.EEF75241E75@unhygienix.ics.uci.edu> archived-at: Fri, 24 Jun 2016 17:02:24 -0000 Michael Blow has submitted this change and it was merged. Change subject: Cleanup Large Page Logic ...................................................................... Cleanup Large Page Logic Change-Id: I26f9a8e4530edfda1b5f78114a5c8038bf8a27f4 Reviewed-on: https://asterix-gerrit.ics.uci.edu/955 Reviewed-by: Jenkins Tested-by: Jenkins Reviewed-by: abdullah alamoudi --- M hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/frames/BTreeNSMLeafFrame.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/VirtualBufferCache.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/CachedPage.java M hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/ICachedPage.java 4 files changed, 27 insertions(+), 16 deletions(-) Approvals: abdullah alamoudi: Looks good to me, approved Jenkins: Looks good to me, but someone else must approve; Verified diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/frames/BTreeNSMLeafFrame.java b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/frames/BTreeNSMLeafFrame.java index 63f516e..f7af7ad 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/frames/BTreeNSMLeafFrame.java +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-btree/src/main/java/org/apache/hyracks/storage/am/btree/frames/BTreeNSMLeafFrame.java @@ -175,10 +175,7 @@ } boolean isLargeTuple(int tupleSize) { - // TODO(mblow): make page size available to avoid calculating it - int pageSize = getLargeFlag() ? buf.capacity() / page.getFrameSizeMultiplier() : buf.capacity(); - - return tupleSize > getMaxTupleSize(pageSize); + return tupleSize > getMaxTupleSize(page.getPageSize()); } @Override @@ -247,9 +244,11 @@ targetFrame = this; } int tuplesToRight = tupleCount - tuplesToLeft; - if (getLargeFlag()) { - ((BTreeNSMLeafFrame) rightFrame).growCapacity(extraPageBlockHelper, bufferCache, - buf.capacity() - rightFrame.getBuffer().capacity()); + + ((BTreeNSMLeafFrame) rightFrame).setLargeFlag(getLargeFlag()); + int deltaPages = page.getFrameSizeMultiplier() - rightFrame.getPage().getFrameSizeMultiplier(); + if (deltaPages > 0) { + ((BTreeNSMLeafFrame) rightFrame).growCapacity(extraPageBlockHelper, bufferCache, deltaPages); } ByteBuffer right = rightFrame.getBuffer(); @@ -298,18 +297,18 @@ public void ensureCapacity(IBufferCache bufferCache, ITupleReference tuple, IExtraPageBlockHelper extraPageBlockHelper) throws HyracksDataException { + // we call ensureCapacity() for large tuples- ensure large flag is set + setLargeFlag(true); int gapBytes = getBytesRequiredToWriteTuple(tuple) - getFreeContiguousSpace(); - growCapacity(extraPageBlockHelper, bufferCache, gapBytes); + if (gapBytes > 0) { + int deltaPages = (int) Math.ceil((double) gapBytes / bufferCache.getPageSize()); + growCapacity(extraPageBlockHelper, bufferCache, deltaPages); + } } - public void growCapacity(IExtraPageBlockHelper extraPageBlockHelper, - IBufferCache bufferCache, int delta) throws HyracksDataException { - setLargeFlag(true); - if (delta <= 0) { - return; - } - int deltaPages = (int) Math.ceil((double) delta / bufferCache.getPageSize()); - int framePagesOld = getBuffer().capacity() / bufferCache.getPageSize(); + private void growCapacity(IExtraPageBlockHelper extraPageBlockHelper, + IBufferCache bufferCache, int deltaPages) throws HyracksDataException { + int framePagesOld = page.getFrameSizeMultiplier(); int newMultiplier = framePagesOld + deltaPages; // we need to get the old slot offsets before we grow diff --git a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/VirtualBufferCache.java b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/VirtualBufferCache.java index 40c7734..c8ce00f 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/VirtualBufferCache.java +++ b/hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/VirtualBufferCache.java @@ -405,6 +405,11 @@ } @Override + public int getPageSize() { + return pageSize; + } + + @Override public int getFrameSizeMultiplier() { return multiplier; } diff --git a/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/CachedPage.java b/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/CachedPage.java index 4541d52..f003649 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/CachedPage.java +++ b/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/CachedPage.java @@ -165,6 +165,11 @@ } @Override + public int getPageSize() { + return pageReplacementStrategy.getPageSize(); + } + + @Override public void setFrameSizeMultiplier(int multiplier) { this.multiplier = multiplier; } diff --git a/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/ICachedPage.java b/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/ICachedPage.java index 8107234..abbe233 100644 --- a/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/ICachedPage.java +++ b/hyracks-fullstack/hyracks/hyracks-storage-common/src/main/java/org/apache/hyracks/storage/common/buffercache/ICachedPage.java @@ -38,5 +38,7 @@ void setQueueInfo(IQueueInfo queueInfo); + int getPageSize(); + int getFrameSizeMultiplier(); } -- To view, visit https://asterix-gerrit.ics.uci.edu/955 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: merged Gerrit-Change-Id: I26f9a8e4530edfda1b5f78114a5c8038bf8a27f4 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Michael Blow Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Michael Blow Gerrit-Reviewer: abdullah alamoudi