hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kiran Kumar.M.R" <Kiran.Kumar...@huawei.com>
Subject RE: Clarifications on HBase Durability
Date Thu, 09 Oct 2014 11:57:20 GMT
Hi Lars,
Thanks for the blog link and tips to enable durable sync at HDFS and Ext.
Will consider these points in our deployment.


Regards,
Kiran
__________________________________________________________________________________________________________
This e-mail and its attachments contain confidential information from HUAWEI, which is intended
only for the person or entity whose address is listed above. Any use of the information contained
herein in any way (including, but not limited to, total or partial disclosure, reproduction,
or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive
this e-mail in error, please notify the sender by phone or email immediately and delete it!
__________________________________________________________________________________________________________




-----Original Message-----
From: lars hofhansl [mailto:larsh@apache.org] 
Sent: Thursday, October 09, 2014 11:37
To: user@hbase.apache.org
Subject: Re: Clarifications on HBase Durability

Correct on all points.
I really need to pick up HBASE-5954 again. What prevented me before was that (a) there was
little interest (even from myself in the end) and (b) with the variety of Hadoop versions
we supported in 0.94 this was reflection hell. In HBase 1.x or 2.x we will likely only support
Hadoop version with HDFS-744 and hence this can be done without reflection there.
Also check out this one: http://hadoop-hbase.blogspot.in/2013/07/protected-hbase-against-data-center.html(It's
even worse, by default you can even lose recently compacted files - and hence _old_ data during
a power outage).
Lastly as discussed in HDFS-5042 you should enable DIR_SYNC on EXT4 or EXT3 if you're using
that on your data nodes.
-- Lars

      From: Kiran Kumar.M.R <Kiran.Kumar.MR@huawei.com>
 To: "user@hbase.apache.org" <user@hbase.apache.org>
 Sent: Tuesday, October 7, 2014 8:19 AM
 Subject: Clarifications on HBase Durability
   
Hi,
We are trying to understand durability possible with Hbase and any contingencies that need
to be planned.

I went through few blogs by Lars (http://bit.ly/10KcvVb , http://bit.ly/1y1sZWH  ) and JIRA
(HBASE-5954, HBASE-12130) This is my understanding:

1.      HBase currently uses flush/hflush interface of HDFS. This ensures that WALEdit
are flushed to all DN (but not flushed to disk)

2.      If there is datacenter level power failure, there are chances of data loss due
to loss of WAL edits

3.      Complete sync behavior will be available only after HBASE-5954 is completed.

Is this understanding correct?


Regards,
Kiran
__________________________________________________________________________________________________________
This e-mail and its attachments contain confidential information from HUAWEI, which is intended
only for the person or entity whose address is listed above. Any use of the information contained
herein in any way (including, but not limited to, total or partial disclosure, reproduction,
or dissemination) by persons other than the intended recipient(s) is prohibited. If you receive
this e-mail in error, please notify the sender by phone or email immediately and delete it!
__________________________________________________________________________________________________________





  
Mime
View raw message