Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; s=serpent; d=yahoo-inc.com; c=nofws; q=dns;
	h=message-id:date:from:user-agent:mime-version:to:subject:
	references:in-reply-to:content-type:content-transfer-encoding;
	b=DkYDoFdbtF6cN4yDPjTn7LLw2BFf/y9+/ZKYTJvp0lrQaR+Uwz4OIf1PEep/k4EG
Message-ID: <4964FC9D.4010405@yahoo-inc.com>
Date: Wed, 07 Jan 2009 11:03:57 -0800
From: Raghu Angadi <rangadi@yahoo-inc.com>
User-Agent: Thunderbird 2.0.0.19 (Windows/20081209)
MIME-Version: 1.0
To: core-user@hadoop.apache.org
Subject: Re: Question about the Namenode edit log and syncing the edit log
 to disk. 0.19.0
References: <4964F804.2010709@attributor.com>
In-Reply-To: <4964F804.2010709@attributor.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit


Did you look at FSEditLog.EditLogFileOutputStream.flushAndSync()?

This code was re-organized sometime back. But the guarantees it provides 
should be exactly same as before. Please let us know otherwise.

Raghu.

Jason Venner wrote:
> I have always assumed (which is clearly my error) that edit log writes 
> were flushed to storage to ensure that the edit log was consistent 
> during machine crash recovery.
> 
> I have been working through FSEditLog.java and I don't see any calls of 
> force(true) on the file channel or sync on the file descriptor, and the 
> edit log is not opened with an 's' or 'd' ie: the open flags are "rw" 
> and not "rws" or "rwd".
> 
> The only thing I see in the code, is that the space in the file where 
> the updates will be written is preallocated.
> 
> Have I missed the mechanism that the edit log data is flushed to the disk?
> 
> Is the edit log data not forcibly flushed to the disk, instead reling on 
> the host operating system to perform the physical writes at a later date?
> 
> Thanks -- Jason