hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <la...@apache.org>
Subject Re: Clarifications on HBase Durability
Date Thu, 09 Oct 2014 06:06:47 GMT
Correct on all points.
I really need to pick up HBASE-5954 again. What prevented me before was that (a) there was
little interest (even from myself in the end) and (b) with the variety of Hadoop versions
we supported in 0.94 this was reflection hell. In HBase 1.x or 2.x we will likely only support
Hadoop version with HDFS-744 and hence this can be done without reflection there.
Also check out this one: http://hadoop-hbase.blogspot.in/2013/07/protected-hbase-against-data-center.html(It's
even worse, by default you can even lose recently compacted files - and hence _old_ data during
a power outage).
Lastly as discussed in HDFS-5042 you should enable DIR_SYNC on EXT4 or EXT3 if you're using
that on your data nodes.
-- Lars

      From: Kiran Kumar.M.R <Kiran.Kumar.MR@huawei.com>
 To: "user@hbase.apache.org" <user@hbase.apache.org> 
 Sent: Tuesday, October 7, 2014 8:19 AM
 Subject: Clarifications on HBase Durability
We are trying to understand durability possible with Hbase and any contingencies that need
to be planned.

I went through few blogs by Lars (http://bit.ly/10KcvVb , http://bit.ly/1y1sZWH  ) and JIRA
(HBASE-5954, HBASE-12130)
This is my understanding:

1.      HBase currently uses flush/hflush interface of HDFS. This ensures that WALEdit
are flushed to all DN (but not flushed to disk)

2.      If there is datacenter level power failure, there are chances of data loss due
to loss of WAL edits

3.      Complete sync behavior will be available only after HBASE-5954 is completed.

Is this understanding correct?

This e-mail and its attachments contain confidential information from HUAWEI, which is intended
only for the person or entity whose address is listed above. Any use of the information contained
herein in any way (including, but not limited to, total or partial disclosure, reproduction,
or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive
this e-mail in error, please notify the sender by phone or email immediately and delete it!

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message