hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Data loss on clean RS shutdown without WAL?
Date Sun, 23 Jan 2011 19:17:02 GMT
The master splits the logs per region before reassigning them. The log
splits are put directly in the region's folder so that when a region
server opens a region that comes from a dead server, it looks for
those files and processes them first before opening the region (to
ensure consistency).

Splitting logs can be slow when you have tons of them, since only one
machine does it, so work is being done to parallelize it just like in
Bigtable: https://issues.apache.org/jira/browse/hbase-1364


On Sun, Jan 23, 2011 at 11:08 AM, M. C. Srivas <mcsrivas@gmail.com> wrote:
> Hey JD,
>   when the RS dies, the regions that it was serving are spread out amongst
> the rest of the RS's, correct?  But isn't the WAL a per-RS thingy rather
> than a per-region thingy? How do the other RS's then recover the regions
> alloted to them? Do they skip over log-records in the dead RS's WAL that do
> not belong to the regions not allocated to them?
>    Also, how is the dead RS's WAL garbage-collected?
> thanks,
> Srivas.

View raw message