Return-Path: 
 <hdfs-dev-return-1202-apmail-hadoop-hdfs-dev-archive=hadoop.apache.org@hadoop.apache.org>
Delivered-To: apmail-hadoop-hdfs-dev-archive@minotaur.apache.org
Received: (qmail 99592 invoked from network); 11 Jun 2010 01:32:48 -0000
Received: from unknown (HELO mail.apache.org) (140.211.11.3)
  by 140.211.11.9 with SMTP; 11 Jun 2010 01:32:48 -0000
Received: (qmail 37989 invoked by uid 500); 11 Jun 2010 01:32:48 -0000
Delivered-To: apmail-hadoop-hdfs-dev-archive@hadoop.apache.org
Received: (qmail 37855 invoked by uid 500); 11 Jun 2010 01:32:47 -0000
Mailing-List: contact hdfs-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:hdfs-dev-help@hadoop.apache.org>
List-Unsubscribe: <mailto:hdfs-dev-unsubscribe@hadoop.apache.org>
List-Post: <mailto:hdfs-dev@hadoop.apache.org>
List-Id: <hdfs-dev.hadoop.apache.org>
Reply-To: hdfs-dev@hadoop.apache.org
Delivered-To: mailing list hdfs-dev@hadoop.apache.org
Received: (qmail 37839 invoked by uid 99); 11 Jun 2010 01:32:47 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 Jun 2010 01:32:47 +0000
X-ASF-Spam-Status: No, hits=1.4 required=10.0
	tests=AWL,HTML_MESSAGE,SPF_NEUTRAL
X-Spam-Check-By: apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Received: from [209.85.214.176] (HELO mail-iw0-f176.google.com)
 (209.85.214.176)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 Jun 2010 01:32:41 +0000
Received: by iwn9 with SMTP id 9so943839iwn.35
        for <multiple recipients>; Thu, 10 Jun 2010 18:32:20 -0700 (PDT)
Received: by 10.231.169.6 with SMTP id w6mr1086287iby.5.1276219939360;
 Thu, 10
	Jun 2010 18:32:19 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.231.161.141 with HTTP; Thu, 10 Jun 2010 18:31:59 -0700 (PDT)
In-Reply-To: <AANLkTilTHDTlxGmxT9UQg-0WymQ9xTQJ-mPJ4KVREPuo@mail.gmail.com>
References: <AANLkTilTHDTlxGmxT9UQg-0WymQ9xTQJ-mPJ4KVREPuo@mail.gmail.com>
From: Todd Lipcon <todd@cloudera.com>
Date: Thu, 10 Jun 2010 18:31:59 -0700
Message-ID: <AANLkTiki6Vnmtf5FgipE4UTlw2wY7vauvwYV1iS97BT7@mail.gmail.com>
Subject: Re: Is it safe enough to write edit log in buffer before writing to
	disk ?
To: hdfs-dev@hadoop.apache.org
Cc: hdfs-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0016e6d26c55240ec80488b71ce3

--0016e6d26c55240ec80488b71ce3
Content-Type: text/plain; charset=ISO-8859-1

Hi Jeff,

All of the FSNamesystem methods call logSync() before returning to the
client. So, if the edit is lost, it also will not have returned a success to
the client.

-Todd

On Thu, Jun 10, 2010 at 6:29 PM, Jeff Zhang <zjffdu@gmail.com> wrote:

> Hi all,
>
> I check the source code of EditLogFileOutputStream, it seems hadoop
> will first write edit log to buffer, then flush to disk. I know that
> it will improve performance, but in the other hand it will cause the
> edit log in buffer lost when the name node is down. So I wonder is it
> possible and necessary to let users do configuration in the conf file
> to decide whether use buffer or not? Thanks.
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>



-- 
Todd Lipcon
Software Engineer, Cloudera

--0016e6d26c55240ec80488b71ce3--