Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id AF2AE200C09 for ; Tue, 10 Jan 2017 19:57:11 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id AD49F160B3D; Tue, 10 Jan 2017 18:57:11 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 8AD42160B2C for ; Tue, 10 Jan 2017 19:57:10 +0100 (CET) Received: (qmail 86645 invoked by uid 500); 10 Jan 2017 18:57:09 -0000 Mailing-List: contact notifications-help@asterixdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@asterixdb.apache.org Delivered-To: mailing list notifications@asterixdb.apache.org Received: (qmail 86636 invoked by uid 99); 10 Jan 2017 18:57:09 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 10 Jan 2017 18:57:09 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 50181180140 for ; Tue, 10 Jan 2017 18:57:09 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.92 X-Spam-Level: X-Spam-Status: No, score=0.92 tagged_above=-999 required=6.31 tests=[SPF_FAIL=0.919, URIBL_BLOCKED=0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id Mxw2qvJbsk8o for ; Tue, 10 Jan 2017 18:57:06 +0000 (UTC) Received: from unhygienix.ics.uci.edu (unhygienix.ics.uci.edu [128.195.14.130]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 719F55F30F for ; Tue, 10 Jan 2017 18:57:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by unhygienix.ics.uci.edu (Postfix) with ESMTP id BCF11241D9F; Tue, 10 Jan 2017 10:57:04 -0800 (PST) Date: Tue, 10 Jan 2017 10:57:03 -0800 From: "Murtadha Hubail (Code Review)" To: Jenkins CC: abdullah alamoudi Reply-To: hubailmor@gmail.com X-Gerrit-MessageType: newpatchset Subject: Change in asterixdb[master]: Introduce Strategy Based Replication and Fault-Tolerance X-Gerrit-Change-Id: I1d1012f5541ce786f127866efefb9f3db434fedd X-Gerrit-ChangeURL: X-Gerrit-Commit: a91d018db11dce89410e36a1bf1335165e1057e8 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Content-Disposition: inline User-Agent: Gerrit/2.8.4 Message-Id: <20170110185704.BCF11241D9F@unhygienix.ics.uci.edu> archived-at: Tue, 10 Jan 2017 18:57:11 -0000 Hello Jenkins, I'd like you to reexamine a change. Please visit https://asterix-gerrit.ics.uci.edu/1405 to look at the new patch set (#10). Change subject: Introduce Strategy Based Replication and Fault-Tolerance ...................................................................... Introduce Strategy Based Replication and Fault-Tolerance This change includes the following: - Introduce new APIs for Replication and Fault-Tolerance Strategies. - Add configuration in cluster description file for high-availability. - Add built-in replication strategies (Metadata_Only, Chained_Declustering) - Add built-in fault-tolerance strategies (Auto, Metadata_Node) - Remove none cluster state functionality from ClusterStateManager. - Add customizable NC startup sequence Change-Id: I1d1012f5541ce786f127866efefb9f3db434fedd --- M asterixdb/asterix-app/src/main/java/org/apache/asterix/api/common/AsterixHyracksIntegrationUtil.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/NCAppRuntimeContext.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/BindMetadataNodeTask.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/CheckpointTask.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/ExternalLibrarySetupTask.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/LocalRecoveryTask.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/MetadataBootstrapTask.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/RemoteRecoveryTask.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/ReportMaxResourceIdTask.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/StartFailbackTask.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/StartLifecycleComponentsTask.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/nc/task/StartReplicationServiceTask.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/AutoFaultToleranceStrategy.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/FaultToleranceStrategyFactory.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/MetadataNodeFaultToleranceStrategy.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/NoFaultToleranceStrategy.java R asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/NodeFailbackPlan.java R asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/CompleteFailbackRequestMessage.java R asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/CompleteFailbackResponseMessage.java C asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/NCLifecycleTaskReportMessage.java R asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/PreparePartitionsFailbackRequestMessage.java R asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/PreparePartitionsFailbackResponseMessage.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/ReplayPartitionLogsRequestMessage.java C asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/ReplayPartitionLogsResponseMessage.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/StartupTaskRequestMessage.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/StartupTaskResponseMessage.java R asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/TakeoverMetadataNodeRequestMessage.java R asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/TakeoverMetadataNodeResponseMessage.java R asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/TakeoverPartitionsRequestMessage.java R asterixdb/asterix-app/src/main/java/org/apache/asterix/app/replication/message/TakeoverPartitionsResponseMessage.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java M asterixdb/asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java A asterixdb/asterix-app/src/main/java/org/apache/asterix/util/FaultToleranceUtil.java M asterixdb/asterix-app/src/main/resources/cluster.xml M asterixdb/asterix-app/src/test/resources/runtimets/results/api/replication/replication.1.adm M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/api/INCLifecycleTask.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/cluster/IClusterStateManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/ClusterProperties.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/config/ReplicationProperties.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/replication/ChainedDeclusteringReplicationStrategy.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/replication/IFaultToleranceStrategy.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/replication/INCLifecycleMessage.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/replication/IRemoteRecoveryManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/replication/IReplicationManager.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/replication/IReplicationStrategy.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/replication/MetadataOnlyReplicationStrategy.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/replication/NoReplicationStrategy.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/replication/Replica.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/replication/ReplicationStrategyFactory.java A asterixdb/asterix-common/src/main/java/org/apache/asterix/common/storage/IndexFileProperties.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/transactions/IRecoveryManager.java M asterixdb/asterix-common/src/main/java/org/apache/asterix/common/utils/StoragePathUtil.java M asterixdb/asterix-common/src/main/resources/schema/cluster.xsd M asterixdb/asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java M asterixdb/asterix-installer/src/main/java/org/apache/asterix/installer/command/ConfigureCommand.java M asterixdb/asterix-installer/src/main/java/org/apache/asterix/installer/command/ValidateCommand.java R asterixdb/asterix-installer/src/main/resources/clusters/local/local_chained_declustering_rep.xml C asterixdb/asterix-installer/src/main/resources/clusters/local/local_metadata_only_rep.xml M asterixdb/asterix-installer/src/test/java/org/apache/asterix/installer/test/AsterixInstallerIntegrationUtil.java A asterixdb/asterix-installer/src/test/java/org/apache/asterix/installer/test/MetadataReplicationIT.java M asterixdb/asterix-installer/src/test/java/org/apache/asterix/installer/test/ReplicationIT.java A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.1.ddl.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.10.node.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.11.sleep.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.12.mgx.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.13.sleep.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.14.ddl.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.15.node.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.16.sleep.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.17.mgx.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.18.sleep.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.19.query.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.2.node.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.3.sleep.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.4.get.http A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.5.mgx.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.6.sleep.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.7.get.http A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.8.query.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/queries/metadata_recovery/metadata_node_recovery/metadata_node_recovery.9.ddl.aql A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/results/metadata_recovery/metadata_node_recovery/metadata_node_recovery.cluster_state.4.adm A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/results/metadata_recovery/metadata_node_recovery/metadata_node_recovery.cluster_state.7.adm A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/results/metadata_recovery/metadata_node_recovery/metadata_node_recovery.query.19.adm A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/results/metadata_recovery/metadata_node_recovery/metadata_node_recovery.query.8.adm A asterixdb/asterix-installer/src/test/resources/integrationts/metadata_only_replication/testsuite.xml M asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/functions/ReplicaFilesRequest.java M asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationChannel.java M asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationManager.java M asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/recovery/RemoteRecoveryManager.java M asterixdb/asterix-replication/src/main/java/org/apache/asterix/replication/storage/ReplicaResourcesManager.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/message/AbstractFailbackPlanMessage.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/util/AppContextInfo.java M asterixdb/asterix-runtime/src/main/java/org/apache/asterix/runtime/util/ClusterStateManager.java M asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java M asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/logging/LogManagerWithReplication.java M asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/recovery/RecoveryManager.java M asterixdb/asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/transaction/TransactionSubsystem.java 99 files changed, 3,764 insertions(+), 1,030 deletions(-) git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/05/1405/10 -- To view, visit https://asterix-gerrit.ics.uci.edu/1405 To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings Gerrit-MessageType: newpatchset Gerrit-Change-Id: I1d1012f5541ce786f127866efefb9f3db434fedd Gerrit-PatchSet: 10 Gerrit-Project: asterixdb Gerrit-Branch: master Gerrit-Owner: Murtadha Hubail Gerrit-Reviewer: Jenkins Gerrit-Reviewer: abdullah alamoudi