hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Josh Elser <els...@apache.org>
Subject Re: WAL Fsync - how do you live without it
Date Tue, 12 Dec 2017 21:08:00 GMT
Redundant power supplies are probably your next-best bet for running 
without fsync (hsync). Something that can prevent a node from going down 
hard will mitigate this issue for the most part.

The importance of this is often a multi-variable equation. The small 
chance for data loss that exists with hflush can be a non-issue for some 
(e.g. ability to replay original data, acceptability of minor data loss, 
knowledge to rebuild hbase:meta as required)

On 12/7/17 6:37 PM, Daniel Połaczański wrote:
> Hi,
> fsync mode on WAL is not supported currenlty. So theoritically it is
> possible that in case of power failure the data is lost because DataNodes
> didn't flush it to physical disk.
> I know that probability is small and data center should have additional
> power source, but still it is possible. How do you deal with it? Do you
> care about it? How to check after the disaster later from which moment the
> data is lost? How often DataNodes flush buffers to disk?
> What is your solution?
> Regards

View raw message