Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of varun@pinterest.com designates
 209.85.210.179 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <1369021773.11235.YahooMailNeo@web140605.mail.bf1.yahoo.com>
References: 
 <CAKxWWm2MAVspehfO-9UU3yA+d+JBKvAv_P3eAsdpBUzzGB37ng@mail.gmail.com>
	<1369021773.11235.YahooMailNeo@web140605.mail.bf1.yahoo.com>
Date: Mon, 20 May 2013 00:35:21 -0700
Message-ID: 
 <CAKxWWm1dD1APaHaAQmDqH=nvbErjqdMaG0ewYze2z5y=cXp6Zw@mail.gmail.com>
Subject: Re: Questions about HBase replication
From: Varun Sharma <varun@pinterest.com>
To: user@hbase.apache.org, lars hofhansl <larsh@apache.org>
Content-Type: multipart/alternative; boundary=047d7bdc07a80b86e404dd216065

--047d7bdc07a80b86e404dd216065
Content-Type: text/plain; charset=ISO-8859-1

Hi Lars,

Thanks for the response.

Regarding #2 again, so if RS1 failed, then the following happens...
1) RS2 takes over its logs...
2) Master renames the log containing directory to have a -splitting in the
path
3) Does RS2 already know about the "-splitting" path ?

Also on a related note, was there a reason that we have all region servers
watching all other region server's queue of logs. Otherwise, couldn't the
master have done the reassignment of outstanding logs to other region
servers more fairly upon failure ?

Thanks
Varun


On Sun, May 19, 2013 at 8:49 PM, lars hofhansl <larsh@apache.org> wrote:

> #1 yes
> #2 no
>
> :)
>
> Now, there are scenarios where inconsistencies can happen. The edits are
> not necessarily shipped in order when there are failures.
> So it is possible to have some Puts at T1 and some Deletes at T2 (T1 <
> T2), and end up with the deletes shipped first.
> Now imagine a compaction happens at the slave after the Deletes are
> shipped to the slave, but before the Puts are shipped... The Puts will
> reappear.
>
> -- Lars
>
>
>
> ________________________________
>  From: Varun Sharma <varun@pinterest.com>
> To: user@hbase.apache.org
> Sent: Sunday, May 19, 2013 12:13 PM
> Subject: Questions about HBase replication
>
>
> Hi,
>
> I have a couple of questions about HBase replication...
>
> 1) When we ship edits to slave cluster - do we retain the timestamps in the
> edits - if we don't, I can imagine hitting some inconsistencies ?
>
> 2) When a region server fails, the master renames the directory containing
> WAL(s). Does this impact reading of those logs for replication ?
>
> Thanks
> Varun
>

--047d7bdc07a80b86e404dd216065--