Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <1556072367.241481266013588050.JavaMail.jira@brutus.apache.org>
Date: Fri, 12 Feb 2010 22:26:28 +0000 (UTC)
From: "ryan rawson (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-2223) Handle 10min+ network partitions
 between clusters
In-Reply-To: <1890224735.238641266004708042.JavaMail.jira@brutus.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-2223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833203#action_12833203 ] 

ryan rawson commented on HBASE-2223:
------------------------------------

the previous plan was to NOT delete the log files if replication needed them... Sounds like you changed that?

If replication needs a log file and the server crashed and they are set to be split, we need to NOT delete the logfile, move them to a holding area perhaps and then get someone to pick the up and send them along.


A 2 hour outage isnt that much, I'd say that we should buffer logs until someone decides it's not worth the disk space.  Ie: make it a top level admin action/alert and give the administrator an option to drop the log retention and then do alternative catch up later.  

It would be better to hold on to a few TB of replication logs then replay that after 24-48 hours of downtime than to mess with the map-reduce stuff, since you'd have to be careful to hopefully avoid duplicating keyvalues.

> Handle 10min+ network partitions between clusters
> -------------------------------------------------
>
>                 Key: HBASE-2223
>                 URL: https://issues.apache.org/jira/browse/HBASE-2223
>             Project: Hadoop HBase
>          Issue Type: Sub-task
>            Reporter: Jean-Daniel Cryans
>            Assignee: Jean-Daniel Cryans
>             Fix For: 0.21.0
>
>
> We need a nice way of handling long network partitions without impacting a master cluster (which pushes the data). Currently it will just retry over and over again.
> I think we could:
>  - Stop replication to a slave cluster if it didn't respond for more than 10 minutes
>  - Keep track of the duration of the partition
>  - When the slave cluster comes back, initiate a MR job like HBASE-2221 
> Maybe we want less than 10 minutes, maybe we want this to be all automatic or just the first 2 parts. Discuss.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.