Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C82F2200D51 for ; Thu, 23 Nov 2017 03:25:39 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id C6614160C0F; Thu, 23 Nov 2017 02:25:39 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 18B2E160BFD for ; Thu, 23 Nov 2017 03:25:38 +0100 (CET) Received: (qmail 85221 invoked by uid 500); 23 Nov 2017 02:25:38 -0000 Mailing-List: contact notifications-help@asterixdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@asterixdb.apache.org Delivered-To: mailing list notifications@asterixdb.apache.org Received: (qmail 85212 invoked by uid 99); 23 Nov 2017 02:25:38 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 23 Nov 2017 02:25:38 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 6439D180710 for ; Thu, 23 Nov 2017 02:25:37 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.919 X-Spam-Level: X-Spam-Status: No, score=0.919 tagged_above=-999 required=6.31 tests=[SPF_FAIL=0.919] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id IUj09ZRQUzYh for ; Thu, 23 Nov 2017 02:25:36 +0000 (UTC) Received: from vitalstatistix.ics.uci.edu (vitalstatistix.ics.uci.edu [128.195.52.38]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 4946C5FD55 for ; Thu, 23 Nov 2017 02:25:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by vitalstatistix.ics.uci.edu (Postfix) with ESMTP id E00CF100ACA; Wed, 22 Nov 2017 18:25:34 -0800 (PST) Date: Wed, 22 Nov 2017 18:25:32 -0800 From: "Luo Chen (Code Review)" To: Jenkins CC: abdullah alamoudi Reply-To: cluo8@uci.edu X-Gerrit-MessageType: newpatchset Subject: Change in asterixdb[master]: [ASTERIXDB-2161] Fix component id manage lifecycle X-Gerrit-Change-Id: Ifc35184c4d431db9af71cab302439e165ee55f54 X-Gerrit-ChangeURL: X-Gerrit-Commit: 70480998cf89df2705994bbd8d5626a6d09540d3 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.12.7 Message-Id: <20171123022534.E00CF100ACA@vitalstatistix.ics.uci.edu> archived-at: Thu, 23 Nov 2017 02:25:40 -0000 Hello Anon. E. Moose #1000171, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2153 to look at the new patch set (#22). Change subject: [ASTERIXDB-2161] Fix component id manage lifecycle ...................................................................... [ASTERIXDB-2161] Fix component id manage lifecycle - user model changes: no - storage format changes: no - interface changes: yes. The interface of LMSIOOperationCallback is changed Details: - The current way of management component ids is not correct, in presence of that multiple partitions sharing the same primary op tracker. It's possible when a partition is empty/being flushed, the next flush is scheduled by another partition, which will disturb the partition. This patch fixes this by using the same logic of maintaining flushed LSNs to maintain component id. - Extend recycle memory component interface to indicate whether it switches the new component or not. - Also fixes [ASTERIXDB-2168] to ensure we do not miss latest flushed LSNs by advancing io callback before finishing flush Change-Id: Ifc35184c4d431db9af71cab302439e165ee55f54 --- M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/ComponentRollbackTest.java M asterixdb/asterix-app/src/test/java/org/apache/asterix/test/dataflow/TestLsmBtreeIoOpCallbackFactory.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/ioopcallbacks/AbstractLSMIOOperationCallback.java A asterixdb/asterix-common/src/test/java/org/apache/asterix/test/ioopcallbacks/AbstractLSMIOOperationCallbackTest.java M asterixdb/asterix-common/src/test/java/org/apache/asterix/test/ioopcallbacks/LSMBTreeIOOperationCallbackTest.java M asterixdb/asterix-common/src/test/java/org/apache/asterix/test/ioopcallbacks/LSMBTreeWithBuddyIOOperationCallbackTest.java M asterixdb/asterix-common/src/test/java/org/apache/asterix/test/ioopcallbacks/LSMInvertedIndexIOOperationCallbackTest.java M asterixdb/asterix-common/src/test/java/org/apache/asterix/test/ioopcallbacks/LSMRTreeIOOperationCallbackTest.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/api/ILSMIOOperationCallback.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/AbstractLSMMemoryComponent.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/BlockingIOOperationCallbackWrapper.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/LSMHarness.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/NoOpIOOperationCallbackFactory.java M hyracks-fullstack/hyracks/hyracks-storage-am-lsm-common/src/main/java/org/apache/hyracks/storage/am/lsm/common/impls/StubIOOperationCallback.java 15 files changed, 414 insertions(+), 306 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/53/2153/22 -- To view, visit https://asterix-gerrit.ics.uci.edu/2153 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ifc35184c4d431db9af71cab302439e165ee55f54 Gerrit-PatchSet: 22 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Luo Chen Gerrit-Reviewer: Anon. E. Moose #1000171 Gerrit-Reviewer: Jenkins Gerrit-Reviewer: abdullah alamoudi