asterixdb-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Murtadha Hubail (Code Review)" <do-not-re...@asterixdb.incubator.apache.org>
Subject Change in asterixdb[master]: Asterix NCs Failback Support
Date Thu, 04 Feb 2016 06:23:55 GMT
Hello Jenkins,

I'd like you to reexamine a change.  Please visit

    https://asterix-gerrit.ics.uci.edu/613

to look at the new patch set (#2).

Change subject: Asterix NCs Failback Support
......................................................................

Asterix NCs Failback Support

This change include the following:
- Allow Failed NCs to failback and takeover their partitions.
- New cluster API servlet for cluster state description.
- Remove nodeId from txn logs except remote FLUSH_LOG.
- Add partition id in UPDATE and Entity_COMMIT logs.
- Adapt remote recovery to new logs format.
- Refactor RecoveryManager and split Analysis and Redo phase.
- Spill remote recover logs to temporary file.
- Prevent logs and files of tookover partitions from being replicated.
- Introduce NC active/inactive partitions concept.
- Test case for failback.

Change-Id: Id17819542d6b9c4e32647e64737c4a467b630f24
---
M asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitPOperator.java
M asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitRuntime.java
M asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/CommitRuntimeFactory.java
M asterix-algebra/src/main/java/org/apache/asterix/algebra/operators/physical/UpsertCommitRuntime.java
M asterix-algebra/src/main/java/org/apache/asterix/optimizer/rules/ReplaceSinkOpWithCommitOpRule.java
M asterix-app/src/main/java/org/apache/asterix/api/common/AsterixAppRuntimeContext.java
A asterix-app/src/main/java/org/apache/asterix/api/http/servlet/ClusterAPIServlet.java
M asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/CCApplicationEntryPoint.java
M asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/ClusterLifecycleListener.java
M asterix-app/src/main/java/org/apache/asterix/hyracks/bootstrap/NCApplicationEntryPoint.java
M asterix-app/src/main/java/org/apache/asterix/messaging/CCMessageBroker.java
M asterix-app/src/main/java/org/apache/asterix/messaging/NCMessageBroker.java
M asterix-common/src/main/java/org/apache/asterix/common/api/IAsterixAppRuntimeContext.java
M asterix-common/src/main/java/org/apache/asterix/common/api/IClusterManagementWork.java
M asterix-common/src/main/java/org/apache/asterix/common/api/IDatasetLifecycleManager.java
M asterix-common/src/main/java/org/apache/asterix/common/context/DatasetLifecycleManager.java
M asterix-common/src/main/java/org/apache/asterix/common/dataflow/AsterixLSMInsertDeleteOperatorNodePushable.java
A asterix-common/src/main/java/org/apache/asterix/common/messaging/AbstractFailbackPlanMessage.java
A asterix-common/src/main/java/org/apache/asterix/common/messaging/CompleteFailbackRequestMessage.java
A asterix-common/src/main/java/org/apache/asterix/common/messaging/CompleteFailbackResponseMessage.java
A asterix-common/src/main/java/org/apache/asterix/common/messaging/PreparePartitionsFailbackRequestMessage.java
A asterix-common/src/main/java/org/apache/asterix/common/messaging/PreparePartitionsFailbackResponseMessage.java
A asterix-common/src/main/java/org/apache/asterix/common/messaging/ReplicaEventMessage.java
M asterix-common/src/main/java/org/apache/asterix/common/messaging/TakeoverPartitionsRequestMessage.java
M asterix-common/src/main/java/org/apache/asterix/common/messaging/api/IApplicationMessage.java
M asterix-common/src/main/java/org/apache/asterix/common/replication/IRemoteRecoveryManager.java
M asterix-common/src/main/java/org/apache/asterix/common/replication/IReplicaResourcesManager.java
M asterix-common/src/main/java/org/apache/asterix/common/replication/IReplicationManager.java
A asterix-common/src/main/java/org/apache/asterix/common/replication/NodeFailbackPlan.java
M asterix-common/src/main/java/org/apache/asterix/common/replication/ReplicaEvent.java
M asterix-common/src/main/java/org/apache/asterix/common/transactions/ILogRecord.java
M asterix-common/src/main/java/org/apache/asterix/common/transactions/IRecoveryManager.java
M asterix-common/src/main/java/org/apache/asterix/common/transactions/LogRecord.java
M asterix-common/src/main/java/org/apache/asterix/common/utils/StoragePathUtil.java
M asterix-common/src/main/java/org/apache/asterix/common/utils/TransactionUtil.java
M asterix-common/src/test/java/org/apache/asterix/test/aql/TestExecutor.java
M asterix-installer/src/test/java/org/apache/asterix/installer/test/ReplicationIT.java
A asterix-installer/src/test/resources/integrationts/replication/queries/failback/node_failback/node_failback.1.ddl.aql
A asterix-installer/src/test/resources/integrationts/replication/queries/failback/node_failback/node_failback.10.cluster_state.aql
A asterix-installer/src/test/resources/integrationts/replication/queries/failback/node_failback/node_failback.11.query.aql
A asterix-installer/src/test/resources/integrationts/replication/queries/failback/node_failback/node_failback.2.update.aql
A asterix-installer/src/test/resources/integrationts/replication/queries/failback/node_failback/node_failback.3.vagrant_script.aql
A asterix-installer/src/test/resources/integrationts/replication/queries/failback/node_failback/node_failback.4.sleep.aql
A asterix-installer/src/test/resources/integrationts/replication/queries/failback/node_failback/node_failback.5.cluster_state.aql
A asterix-installer/src/test/resources/integrationts/replication/queries/failback/node_failback/node_failback.6.query.aql
A asterix-installer/src/test/resources/integrationts/replication/queries/failback/node_failback/node_failback.7.update.aql
A asterix-installer/src/test/resources/integrationts/replication/queries/failback/node_failback/node_failback.8.vagrant_mgx.aql
A asterix-installer/src/test/resources/integrationts/replication/queries/failback/node_failback/node_failback.9.sleep.aql
M asterix-installer/src/test/resources/integrationts/replication/queries/failover/bulkload/bulkload.2.update.aql
M asterix-installer/src/test/resources/integrationts/replication/queries/failover/mem_component_recovery/mem_component_recovery.2.update.aql
A asterix-installer/src/test/resources/integrationts/replication/results/failback/node_failback/node_failback.cluster_state.10.adm
A asterix-installer/src/test/resources/integrationts/replication/results/failback/node_failback/node_failback.cluster_state.5.adm
A asterix-installer/src/test/resources/integrationts/replication/results/failback/node_failback/node_failback.query.11.adm
A asterix-installer/src/test/resources/integrationts/replication/results/failback/node_failback/node_failback.query.6.adm
M asterix-installer/src/test/resources/integrationts/replication/testsuite.xml
M asterix-metadata/src/main/java/org/apache/asterix/metadata/MetadataNode.java
M asterix-om/src/main/java/org/apache/asterix/om/util/AsterixClusterProperties.java
M asterix-replication/src/main/java/org/apache/asterix/replication/functions/ReplicaFilesRequest.java
M asterix-replication/src/main/java/org/apache/asterix/replication/functions/ReplicationProtocol.java
D asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicaEventNotifier.java
M asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationChannel.java
D asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationLifecycleListener.java
M asterix-replication/src/main/java/org/apache/asterix/replication/management/ReplicationManager.java
M asterix-replication/src/main/java/org/apache/asterix/replication/recovery/RemoteRecoveryManager.java
M asterix-replication/src/main/java/org/apache/asterix/replication/storage/LSMIndexFileProperties.java
M asterix-replication/src/main/java/org/apache/asterix/replication/storage/ReplicaResourcesManager.java
M asterix-runtime/src/main/java/org/apache/asterix/runtime/operators/AsterixLSMPrimaryUpsertOperatorNodePushable.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/AbstractIndexModificationOperationCallback.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/PrimaryIndexModificationOperationCallback.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/PrimaryIndexModificationOperationCallbackFactory.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/SecondaryIndexModificationOperationCallback.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/SecondaryIndexModificationOperationCallbackFactory.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/TempDatasetIndexModificationOperationCallback.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/TempDatasetPrimaryIndexModificationOperationCallbackFactory.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/TempDatasetSecondaryIndexModificationOperationCallbackFactory.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/UpsertOperationCallback.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/opcallbacks/UpsertOperationCallbackFactory.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/resource/PersistentLocalResourceRepository.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/logging/LogManagerWithReplication.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/logging/LogReader.java
A asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/logging/RemoteLogReader.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/recovery/RecoveryManager.java
A asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/recovery/TxnId.java
M asterix-transactions/src/main/java/org/apache/asterix/transaction/management/service/transaction/TransactionContext.java
84 files changed, 3,139 insertions(+), 2,120 deletions(-)


  git pull ssh://asterix-gerrit.ics.uci.edu:29418/asterixdb refs/changes/13/613/2
-- 
To view, visit https://asterix-gerrit.ics.uci.edu/613
To unsubscribe, visit https://asterix-gerrit.ics.uci.edu/settings

Gerrit-MessageType: newpatchset
Gerrit-Change-Id: Id17819542d6b9c4e32647e64737c4a467b630f24
Gerrit-PatchSet: 2
Gerrit-Project: asterixdb
Gerrit-Branch: master
Gerrit-Owner: Murtadha Hubail <hubailmor@gmail.com>
Gerrit-Reviewer: Jenkins <jenkins@fulliautomatix.ics.uci.edu>

Mime
View raw message