hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Data loss on clean RS shutdown without WAL?
Date Fri, 21 Jan 2011 17:32:27 GMT
If the region servers gets YouAreDeadException, it does an "abort" and
won't flush the data since another region server could already be
serving the region. If you're not writing to the WAL, then yes it's
data loss.

Not sure what you mean by "shuts down cleanly" in your case, if you
see a log that starts with "Aborting region server" then it's not
really "clean".

J-D

On Fri, Jan 21, 2011 at 2:38 AM, Friso van Vollenhoven
<fvanvollenhoven@xebia.com> wrote:
> Hi all,
>
> Question: when a regionserver shuts down cleanly after a YouAreDeadException and the
master nicely reassigns the regions, will you loose any data that was written to the memstore
of the dead RS when not using WAL?
>
> There was no hard crash and not a single error in any of the logs (except for the FATAL:
YouAreDeadException). The RS lost its zookeeper session after a timeout, probably GC combined
with some other starvation on heavy load. I think the memstore flushes on shutdown, but I
am not entirely sure what happens in the situation where regions are already opened by other
regionservers when the dying executes the shutdown code. Can I assume that the RS that gets
reassigned a region creates a new HFile and that this will be compacted together with the
one left by the dead RS at the next compaction run?
>
>
> Thanks,
> Friso
>
>

Mime
View raw message