Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 24084200D30 for ; Mon, 30 Oct 2017 21:17:49 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 22886160BF8; Mon, 30 Oct 2017 20:17:49 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 693E81609D5 for ; Mon, 30 Oct 2017 21:17:48 +0100 (CET) Received: (qmail 5758 invoked by uid 500); 30 Oct 2017 20:17:47 -0000 Mailing-List: contact notifications-help@asterixdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@asterixdb.apache.org Delivered-To: mailing list notifications@asterixdb.apache.org Received: (qmail 5748 invoked by uid 99); 30 Oct 2017 20:17:47 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Oct 2017 20:17:47 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 95C44CD891 for ; Mon, 30 Oct 2017 20:17:46 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.919 X-Spam-Level: X-Spam-Status: No, score=0.919 tagged_above=-999 required=6.31 tests=[SPF_FAIL=0.919] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id hPzVFNmAxT_H for ; Mon, 30 Oct 2017 20:17:45 +0000 (UTC) Received: from vitalstatistix.ics.uci.edu (vitalstatistix.ics.uci.edu [128.195.52.38]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id 8B90E5F121 for ; Mon, 30 Oct 2017 20:17:45 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by vitalstatistix.ics.uci.edu (Postfix) with ESMTP id 3A4B11007D9; Mon, 30 Oct 2017 13:17:45 -0700 (PDT) Date: Mon, 30 Oct 2017 13:17:43 -0700 From: "Murtadha Hubail (Code Review)" To: Jenkins Reply-To: mhubail@apache.org X-Gerrit-MessageType: newpatchset Subject: Change in asterixdb[master]: [ASTERIXDB-2081][STO] Introduce DatasetMemoryManager X-Gerrit-Change-Id: Ica76b3c8eca6f7d2ad1d962fb5ef84267c258571 X-Gerrit-ChangeURL: X-Gerrit-Commit: c28e42759316c7b159126c3444fd14d3ce605248 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.12.7 Message-Id: <20171030201745.3A4B11007D9@vitalstatistix.ics.uci.edu> archived-at: Mon, 30 Oct 2017 20:17:49 -0000 Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/2112 to look at the new patch set (#3). Change subject: [ASTERIXDB-2081][STO] Introduce DatasetMemoryManager ...................................................................... [ASTERIXDB-2081][STO] Introduce DatasetMemoryManager - user model changes: no - storage format changes: no - interface changes: yes Added IDatasetMemoryManager to manage datasets memory reservation and allocation. Details: - Reserve metadata datasets memory to allow them to be opened when needed. - Close all datasets at the end of recovery to clear used memory and flush all recovered datasets. - Add UngracefulShutdownNCApplication to force recovery to run on AsterixHyracksIntegrationUtil. - Refactor the use of firstAvilableUserDatasetID to check for metadata datasets. - Add ThreadSafe annotation. - Add test case for RecoveryManager after creating multiple datasets. Change-Id: Ica76b3c8eca6f7d2ad1d962fb5ef84267c258571 --- M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/NCAppRuntimeContext.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/RecoveryManager.java A asterixdb/asterix-app/src/test/java/org/apache/asterix/test/txn/RecoveryManagerTest.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetMemoryManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/INcApplicationContext.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetMemoryManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetResource.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetVirtualBufferCaches.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/metadata/MetadataIndexImmutableProperties.java A asterixdb/asterix-common/src/test/java/org/apache/asterix/test/context/DatasetMemoryManagerTest.java M asterixdb/asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java A hyracks-fullstack/hyracks/hyracks-util/src/main/java/org/apache/hyracks/util/annotations/ThreadSafe.java 14 files changed, 563 insertions(+), 65 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/12/2112/3 -- To view, visit https://asterix-gerrit.ics.uci.edu/2112 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ica76b3c8eca6f7d2ad1d962fb5ef84267c258571 Gerrit-PatchSet: 3 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins