hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@hortonworks.com>
Subject Re: Hadoop hardware failure recovery
Date Mon, 13 Aug 2012 15:08:01 GMT
On 13 August 2012 07:55, Harsh J <harsh@cloudera.com> wrote:


Note that with 2.1.0 (upcoming) and above releases of HDFS, we offer a
> working hsync() API that allows you to write files with guarantee that
> it has been written to the disk (like the fsync() *nix call).

A guarantee that the OS thinks it's been written to HDD.

For anyone using Hadoop or any other program (e.g MySQL) in a virtualized
environment , even when the OS thinks it has flushed a virtual disk -know
that you may have set some VM params to say "when we said "flush to disk"
we meant it":

View raw message