hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raghu Angadi <rang...@yahoo-inc.com>
Subject Re: Question about the Namenode edit log and syncing the edit log to disk. 0.19.0
Date Wed, 07 Jan 2009 19:03:57 GMT

Did you look at FSEditLog.EditLogFileOutputStream.flushAndSync()?

This code was re-organized sometime back. But the guarantees it provides 
should be exactly same as before. Please let us know otherwise.


Jason Venner wrote:
> I have always assumed (which is clearly my error) that edit log writes 
> were flushed to storage to ensure that the edit log was consistent 
> during machine crash recovery.
> I have been working through FSEditLog.java and I don't see any calls of 
> force(true) on the file channel or sync on the file descriptor, and the 
> edit log is not opened with an 's' or 'd' ie: the open flags are "rw" 
> and not "rws" or "rwd".
> The only thing I see in the code, is that the space in the file where 
> the updates will be written is preallocated.
> Have I missed the mechanism that the edit log data is flushed to the disk?
> Is the edit log data not forcibly flushed to the disk, instead reling on 
> the host operating system to perform the physical writes at a later date?
> Thanks -- Jason

View raw message