hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Duo Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-19397) Design procedures for ReplicationManager to notify peer change event from master
Date Sat, 06 Jan 2018 03:44:00 GMT

     [ https://issues.apache.org/jira/browse/HBASE-19397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Duo Zhang updated HBASE-19397:
------------------------------
    Release Note: 
Introduce 5 procedures to do peer modifications:
AddPeerProcedure
RemovePeerProcedure
UpdatePeerConfigProcedure
EnablePeerProcedure
DisablePeerProcedure

The procedures are all executed with the following stage:
1. Call pre CP hook, if an exception is thrown then give up
2. Check whether the operation is valid, if not then give up
3. Update peer storage. Notice that if we have entered this stage, then we can not rollback
any more.
4. Schedule sub procedures to refresh the peer config on every RS.
5. Do post cleanup if any.
6. Call post CP hook. The exception thrown will be ignore since we have already done the work.

The procedure will hold an exclusive lock on the peer id, so now there is no concurrent modifications
on a single peer.

And now it is guaranteed that once the procedure is done, the peer modification has already
taken effect on all RSes.

Abstracte a storage layer for replication peer/queue manangement, and refactored the upper
layer to remove zk related naming/code/comment.

Add pre/postExecuteProcedures CP hooks to RegionServerObserver, and add permission check for
executeProcedures method which requires the caller to be system user or super user.

On rolling upgrade: just do not do any replication peer modifications during the rolling upgrading.
There is no pb/layout changes on the peer/queue storage on zk.

  was:
Introduce 5 procedures to do peer modifications:
AddPeerProcedure
RemovePeerProcedure
UpdatePeerConfigProcedure
EnablePeerProcedure
DisablePeerProcedure

The procedures are all executed with the following stage:
1. Call pre CP hook, if an exception is thrown then give up
2. Check whether the operation is valid, if not then give up
3. Update peer storage. Notice that if we have entered this stage, then we can not rollback
any more.
4. Schedule sub procedures to refresh the peer config on every RS.
5. Do post cleanup if any.
6. Call post CP hook. The exception thrown will be ignore since we have already done the work.

The procedure will hold an exclusive lock on the peer id, so now there is no concurrent modifications
on a single peer.

And now it is guaranteed that once the procedure is done, the peer modification has already
taken effect on all RSes.

Abstracte a storage layer for replication peer/queue manangement, and refactored the upper
layer to remove zk related naming/code/comment.

Add pre/postExecuteProcedures CP hooks to RegionServerObserver, and add permission check for
executeProcedures method which requires the caller to be system user or super user.


> Design  procedures for ReplicationManager to notify peer change event from master
> ---------------------------------------------------------------------------------
>
>                 Key: HBASE-19397
>                 URL: https://issues.apache.org/jira/browse/HBASE-19397
>             Project: HBase
>          Issue Type: New Feature
>          Components: proc-v2, Replication
>            Reporter: Zheng Hu
>            Assignee: Zheng Hu
>         Attachments: HBASE-19397-master-v1.patch, HBASE-19397-master.patch
>
>
> After we store peer states / peer queues information into hbase table,   RS can not track
peer config change by adding watcher znode.   
> So we need design procedures for ReplicationManager to notify peer change event.   the
replication rpc interfaces which may be implemented by procedures are following: 
> {code}
> 1. addReplicationPeer
> 2. removeReplicationPeer
> 3. enableReplicationPeer
> 4. disableReplicationPeer
> 5. updateReplicationPeerConfig
> {code}
> BTW,  our RS states will still be store in zookeeper,  so when RS crash, the tracker
which will trigger to transfer queues of crashed RS will still be a Zookeeper Tracker.  we
need NOT implement that by  procedures.  
> As we will  release 2.0 in next weeks,  and the HBASE-15867 can not be resolved before
the release,  so I'd prefer to create a new feature branch for HBASE-15867. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message