hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-14379) Replication V2
Date Tue, 08 Sep 2015 05:00:49 GMT

     [ https://issues.apache.org/jira/browse/HBASE-14379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Purtell updated HBASE-14379:
-----------------------------------
    Description: 
Replication V2 is a tear-down of exiting replication code to just the interfaces introduced
in HBASE-11367, then a rebuild around the following principles, goals, and suggested features:
- No state in ZooKeeper. Introduce a new system table for tracking peers, queues, and log
positions. (Some discussion on HBASE-10295, probably will be replaced with a set of more focused
issues.)
- Allow replication v1 and v2 to coexist. Note all of the undesirable features of v1 will
remain as long as v1 is active, 'fixing' v1 is out of scope.
- Simplified internal programming model based on iterators
- Streaming data transfer
- Administrative actions mediated by the master with support for security hooks (like HBASE-11392)
- Replication state persisted and communicated with protobuf (like HBASE-11393 but everywhere)
- Detailed metrics
- Support for at least simple status checks and admin actions via UI and shell
- Hbck support for fixing corrupt or stuck queues (like HBASE-14014)
- Support for bulk load, perhaps through augmenting bulk load to build WALs as well as HFiles
(see HBASE-13153)
- Optional consideration for replicating schema as well as data (like HBASE-12947). May fall
out of scope.
- Optional separation of replication function from the regionservers (see HBASE-8772)
- Optional alternate scheduling of edits besides FIFO-by-region (see HBASE-1734 and HBASE-14014)

There are a number of existing JIRAs that will eventually be closed as duplicate, wont fix,
or reparented here.

  was:
Replication V2 is a tear-down of exiting replication code to just the interfaces introduced
in HBASE-11367, then a rebuild around the following principles, goals, and suggested features:
- No state in ZooKeeper. Introduce a new system table for tracking peers, queues, and log
positions. (Some discussion on HBASE-10295, probably will be replaced with a set of more focused
issues.)
- Simplified internal programming model based on iterators
- Streaming data transfer
- Administrative actions mediated by the master with support for security hooks (like HBASE-11392)
- Replication state persisted and communicated with protobuf (like HBASE-11393 but everywhere)
- Detailed metrics
- Support for at least simple status checks and admin actions via UI and shell
- Hbck support for fixing corrupt or stuck queues (like HBASE-14014)
- Support for bulk load, perhaps through augmenting bulk load to build WALs as well as HFiles
(see HBASE-13153)
- Optional consideration for replicating schema as well as data (like HBASE-12947). May fall
out of scope.
- Optional separation of replication function from the regionservers (see HBASE-8772)
- Optional alternate scheduling of edits besides FIFO-by-region (see HBASE-1734 and HBASE-14014)

There are a number of existing JIRAs that will eventually be closed as duplicate, wont fix,
or reparented here.


> Replication V2
> --------------
>
>                 Key: HBASE-14379
>                 URL: https://issues.apache.org/jira/browse/HBASE-14379
>             Project: HBase
>          Issue Type: Umbrella
>          Components: Replication
>            Reporter: Andrew Purtell
>             Fix For: 2.0.0
>
>
> Replication V2 is a tear-down of exiting replication code to just the interfaces introduced
in HBASE-11367, then a rebuild around the following principles, goals, and suggested features:
> - No state in ZooKeeper. Introduce a new system table for tracking peers, queues, and
log positions. (Some discussion on HBASE-10295, probably will be replaced with a set of more
focused issues.)
> - Allow replication v1 and v2 to coexist. Note all of the undesirable features of v1
will remain as long as v1 is active, 'fixing' v1 is out of scope.
> - Simplified internal programming model based on iterators
> - Streaming data transfer
> - Administrative actions mediated by the master with support for security hooks (like
HBASE-11392)
> - Replication state persisted and communicated with protobuf (like HBASE-11393 but everywhere)
> - Detailed metrics
> - Support for at least simple status checks and admin actions via UI and shell
> - Hbck support for fixing corrupt or stuck queues (like HBASE-14014)
> - Support for bulk load, perhaps through augmenting bulk load to build WALs as well as
HFiles (see HBASE-13153)
> - Optional consideration for replicating schema as well as data (like HBASE-12947). May
fall out of scope.
> - Optional separation of replication function from the regionservers (see HBASE-8772)
> - Optional alternate scheduling of edits besides FIFO-by-region (see HBASE-1734 and HBASE-14014)
> There are a number of existing JIRAs that will eventually be closed as duplicate, wont
fix, or reparented here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message