hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-9047) Tool to handle finishing replication when the cluster is offline
Date Wed, 30 Oct 2013 22:13:26 GMT

    [ https://issues.apache.org/jira/browse/HBASE-9047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13809694#comment-13809694
] 

Hadoop QA commented on HBASE-9047:
----------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12611148/HBASE-9047-trunk-v3.patch
  against trunk revision .

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 2 new or modified
tests.

    {color:green}+1 hadoop1.0{color}.  The patch compiles against the hadoop 1.0 profile.

    {color:green}+1 hadoop2.0{color}.  The patch compiles against the hadoop 2.0 profile.

    {color:green}+1 javadoc{color}.  The javadoc tool did not generate any warning messages.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:red}-1 findbugs{color}.  The patch appears to introduce 1 new Findbugs (version
1.3.9) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 lineLengths{color}.  The patch does not introduce lines longer than 100

    {color:red}-1 site{color}.  The patch appears to cause mvn site goal to fail.

    {color:green}+1 core tests{color}.  The patch passed unit tests in .

Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7679//testReport/
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7679//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7679//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7679//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7679//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7679//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7679//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7679//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7679//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7679//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7679//console

This message is automatically generated.

> Tool to handle finishing replication when the cluster is offline
> ----------------------------------------------------------------
>
>                 Key: HBASE-9047
>                 URL: https://issues.apache.org/jira/browse/HBASE-9047
>             Project: HBase
>          Issue Type: New Feature
>    Affects Versions: 0.96.0
>            Reporter: Jean-Daniel Cryans
>            Assignee: Demai Ni
>             Fix For: 0.98.0
>
>         Attachments: HBASE-9047-0.94.9-v0.PATCH, HBASE-9047-trunk-v0.patch, HBASE-9047-trunk-v1.patch,
HBASE-9047-trunk-v2.patch, HBASE-9047-trunk-v3.patch
>
>
> We're having a discussion on the mailing list about replicating the data on a cluster
that was shut down in an offline fashion. The motivation could be that you don't want to bring
HBase back up but still need that data on the slave.
> So I have this idea of a tool that would be running on the master cluster while it is
down, although it could also run at any time. Basically it would be able to read the replication
state of each master region server, finish replicating what's missing to all the slave, and
then clear that state in zookeeper.
> The code that handles replication does most of that already, see ReplicationSourceManager
and ReplicationSource. Basically when ReplicationSourceManager.init() is called, it will check
all the queues in ZK and try to grab those that aren't attached to a region server. If the
whole cluster is down, it will grab all of them.
> The beautiful thing here is that you could start that tool on all your machines and the
load will be spread out, but that might not be a big concern if replication wasn't lagging
since it would take a few seconds to finish replicating the missing data for each region server.
> I'm guessing when starting ReplicationSourceManager you'd give it a fake region server
ID, and you'd tell it not to start its own source.
> FWIW the main difference in how replication is handled between Apache's HBase and Facebook's
is that the latter is always done separately of HBase itself. This jira isn't about doing
that.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message