Return-Path: X-Original-To: apmail-asterixdb-notifications-archive@minotaur.apache.org Delivered-To: apmail-asterixdb-notifications-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 1462C18329 for ; Thu, 31 Dec 2015 16:38:40 +0000 (UTC) Received: (qmail 7587 invoked by uid 500); 31 Dec 2015 16:38:40 -0000 Delivered-To: apmail-asterixdb-notifications-archive@asterixdb.apache.org Received: (qmail 7553 invoked by uid 500); 31 Dec 2015 16:38:40 -0000 Mailing-List: contact notifications-help@asterixdb.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@asterixdb.incubator.apache.org Delivered-To: mailing list notifications@asterixdb.incubator.apache.org Received: (qmail 7544 invoked by uid 99); 31 Dec 2015 16:38:39 -0000 Received: from Unknown (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Dec 2015 16:38:39 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 8790A1A04CB for ; Thu, 31 Dec 2015 16:38:39 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.92 X-Spam-Level: X-Spam-Status: No, score=0.92 tagged_above=-999 required=6.31 tests=[SPF_FAIL=0.919, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-us-west.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id EOSS9Sm5nKGs for ; Thu, 31 Dec 2015 16:38:35 +0000 (UTC) Received: from unhygienix.ics.uci.edu (unhygienix.ics.uci.edu [128.195.14.130]) by mx1-us-west.apache.org (ASF Mail Server at mx1-us-west.apache.org) with ESMTP id 522202050E for ; Thu, 31 Dec 2015 16:38:35 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by unhygienix.ics.uci.edu (Postfix) with ESMTP id 2DBA82411F4; Thu, 31 Dec 2015 08:34:51 -0800 (PST) Date: Thu, 31 Dec 2015 08:34:51 -0800 From: "Murtadha Hubail (Code Review)" To: abdullah alamoudi , Jenkins CC: Yingyi Bu , Ian Maxon Reply-To: hubailmor@gmail.com X-Gerrit-MessageType: newpatchset Subject: Change in asterixdb[master]: Divide Cluster into Unique Partitions X-Gerrit-Change-Id: I8c7fbca5113dd7ad569a46dfa2591addb5bf8655 X-Gerrit-ChangeURL: X-Gerrit-Commit: 3cfda908f76f45657ef52254d4d547afb1e3624b In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.8.4 Message-Id: <20151231163451.2DBA82411F4@unhygienix.ics.uci.edu> Hello abdullah alamoudi, Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/564 to look at the new patch set (#5). Change subject: Divide Cluster into Unique Partitions ...................................................................... Divide Cluster into Unique Partitions The change includes the following: - Fix passing NC stores to AsterixConfiguration. - Unify storage direcotry name in the instance level rather than the node level. - Divide the cluster into unique storage partitions based on the number of stores. - Refactored FileSplits and moved out of AqlMetadataProvider. - Make AsterixHyracksIntegrationUtil use the passed configuration file. - Make File Splits pass relative index paths of partitions rather than absolute paths. - Remove unused AqlCompiledMetadataDeclarations class. Change-Id: I8c7fbca5113dd7ad569a46dfa2591addb5bf8655 --- M asterix-app/src/main/java/org/apache/asterix/api/common/AsterixAppRuntimeContext.java M asterix-app/src/main/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java M asterix-app/src/main/java/org/apache/asterix/aql/translator/QueryTranslator.java M asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java M asterix-app/src/main/resources/asterix-build-configuration.xml M asterix-app/src/test/java/org/apache/asterix/test/metadata/MetadataTest.java M asterix-app/src/test/java/org/apache/asterix/test/runtime/ExecutionTest.java M asterix-app/src/test/java/org/apache/asterix/test/runtime/SqlppExecutionTest.java A asterix-common/src/main/java/org/apache/asterix/common/cluster/ClusterPartition.java M asterix-common/src/main/java/org/apache/asterix/common/config/AsterixMetadataProperties.java M asterix-common/src/main/java/org/apache/asterix/common/config/AsterixPropertiesAccessor.java M asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java M asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java M asterix-common/src/main/resources/schema/cluster.xsd M asterix-common/src/main/resources/schema/yarn_cluster.xsd M asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java M asterix-events/src/main/java/org/apache/asterix/event/driver/EventDriver.java M asterix-events/src/main/java/org/apache/asterix/event/management/EventUtil.java M asterix-events/src/main/java/org/apache/asterix/event/service/AsterixEventServiceUtil.java M asterix-events/src/main/java/org/apache/asterix/event/util/PatternCreator.java M asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/HDFSAdapterFactory.java M asterix-external-data/src/main/java/org/apache/asterix/external/adapter/factory/HDFSIndexingAdapterFactory.java M asterix-external-data/src/main/java/org/apache/asterix/external/indexing/operators/ExternalDatasetIndexesAbortOperatorDescriptor.java M asterix-external-data/src/main/java/org/apache/asterix/external/indexing/operators/ExternalDatasetIndexesCommitOperatorDescriptor.java M asterix-external-data/src/main/java/org/apache/asterix/external/indexing/operators/ExternalDatasetIndexesRecoverOperatorDescriptor.java M asterix-installer/src/main/java/org/apache/asterix/installer/command/ValidateCommand.java M asterix-installer/src/main/java/org/apache/asterix/installer/driver/InstallerUtil.java M asterix-metadata/src/main/java/org/apache/asterix/metadata/bootstrap/MetadataBootstrap.java D asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlCompiledMetadataDeclarations.java M asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlLogicalPlanAndMetadataImpl.java M asterix-metadata/src/main/java/org/apache/asterix/metadata/declared/AqlMetadataProvider.java A asterix-metadata/src/main/java/org/apache/asterix/metadata/utils/SplitsAndConstraintsUtil.java M asterix-om/src/main/java/org/apache/asterix/om/util/AsterixClusterProperties.java M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/PrimaryIndexModificationOperationCallbackFactory.java M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/SecondaryIndexModificationOperationCallbackFactory.java M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/TempDatasetPrimaryIndexModificationOperationCallbackFactory.java M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/TempDatasetSecondaryIndexModificationOperationCallbackFactory.java M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceFactory.java M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/recovery/RecoveryManager.java M asterix-yarn/src/main/java/org/apache/asterix/aoya/AsterixApplicationMaster.java M asterix-yarn/src/main/java/org/apache/asterix/aoya/AsterixYARNClient.java 42 files changed, 844 insertions(+), 941 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/64/564/5 -- To view, visit https://asterix-gerrit.ics.uci.edu/564 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8c7fbca5113dd7ad569a46dfa2591addb5bf8655 Gerrit-PatchSet: 5 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Ian Maxon Gerrit-Reviewer: Jenkins Gerrit-Reviewer: Murtadha Hubail Gerrit-Reviewer: Yingyi Bu Gerrit-Reviewer: abdullah alamoudi