hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8751) Enable peer cluster to choose/change the ColumnFamilies/Tables it really want to replicate from a source cluster
Date Tue, 28 Jan 2014 05:45:41 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13883789#comment-13883789
] 

Hadoop QA commented on HBASE-8751:
----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12625505/HBASE-8751-trunk_v2.patch
  against trunk revision .
  ATTACHMENT ID: 12625505

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 3 new or modified
tests.

    {color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 1.0 profile.

    {color:green}+1 hadoop1.1{color}.  The patch compiles against the hadoop 1.1 profile.

    {color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 4 warning messages.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

  {color:green}+1 site{color}.  The mvn site goal succeeds with this patch.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/8533//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8533//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8533//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8533//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8533//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8533//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8533//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8533//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8533//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/8533//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8533//console

This message is automatically generated.

> Enable peer cluster to choose/change the ColumnFamilies/Tables it really want to replicate
from a source cluster
> ----------------------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-8751
>                 URL: https://issues.apache.org/jira/browse/HBASE-8751
>             Project: HBase
>          Issue Type: New Feature
>          Components: Replication
>            Reporter: Feng Honghua
>            Assignee: Feng Honghua
>         Attachments: HBASE-8751-0.94-V0.patch, HBASE-8751-0.94-v1.patch, HBASE-8751-trunk_v0.patch,
HBASE-8751-trunk_v1.patch, HBASE-8751-trunk_v2.patch, HBASE-8751-trunk_v3.patch
>
>
> Consider scenarios (all cf are with replication-scope=1):
> 1) cluster S has 3 tables, table A has cfA,cfB, table B has cfX,cfY, table C has cf1,cf2.
> 2) cluster X wants to replicate table A : cfA, table B : cfX and table C from cluster
S.
> 3) cluster Y wants to replicate table B : cfY, table C : cf2 from cluster S.
> Current replication implementation can't achieve this since it'll push the data of all
the replicatable column-families from cluster S to all its peers, X/Y in this scenario.
> This improvement provides a fine-grained replication theme which enable peer cluster
to choose the column-families/tables they really want from the source cluster:
> A). Set the table:cf-list for a peer when addPeer:
>   hbase-shell> add_peer '3', "zk:1100:/hbase", "table1; table2:cf1,cf2; table3:cf2"
> B). View the table:cf-list config for a peer using show_peer_tableCFs:
>   hbase-shell> show_peer_tableCFs "1"
> C). Change/set the table:cf-list for a peer using set_peer_tableCFs:
>   hbase-shell> set_peer_tableCFs '2', "table1:cfX; table2:cf1; table3:cf1,cf2"
> In this theme, replication-scope=1 only means a column-family CAN be replicated to other
clusters, but only the 'table:cf-list list' determines WHICH cf/table will actually be replicated
to a specific peer.
> To provide back-compatibility, empty 'table:cf-list list' will replicate all replicatable
cf/table. (this means we don't allow a peer which replicates nothing from a source cluster,
we think it's reasonable: if replicating nothing why bother adding a peer?)
> This improvement addresses the exact problem raised  by the first FAQ in "http://hbase.apache.org/replication.html":
>   "GLOBAL means replicate? Any provision to replicate only to cluster X and not to cluster
Y? or is that for later?
>   Yes, this is for much later."
> I also noticed somebody mentioned "replication-scope" as integer rather than a boolean
is for such fine-grained replication purpose, but I think extending "replication-scope" can't
achieve the same replication granularity flexibility as providing above per-peer replication
configurations.
> This improvement has been running smoothly in our production clusters (Xiaomi) for several
months.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message