Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of dhruba@gmail.com designates
 209.85.128.189 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=message-id:date:from:to:subject:in-reply-to:mime-version
         :content-type:content-transfer-encoding:content-disposition
         :references;
        b=SPbDurAJ3H7S7FL6kZgxacK6Ek4Vf38/T5CmHWwEHki2hp913fZJHLCRNpi1zNh3IG
         2UuQrW3e/e/1kfSoKMGAEO9Ah+4d1/Bsz9WvHMPYY48W/ihyrasD4zH1nZCjDEGJmefg
         uumbpxbd+fSF7iQqw+EvtphAkktrXxWMb65Z0=
Message-ID: <4aa34eb70807312315k314a3a5j7260fb931897c7cf@mail.gmail.com>
Date: Thu, 31 Jul 2008 22:15:34 -0800
From: "Dhruba Borthakur" <dhruba@gmail.com>
To: core-dev@hadoop.apache.org
Subject: Re: HDFS : Question on INodeFileUnderConstruction targets filed ..
In-Reply-To: <811fc3830807311327g40ec854x3acf0ce2eb791441@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
References: <811fc3830807311318r20702652s2b4f99cc22dce6e7@mail.gmail.com>
	 <811fc3830807311327g40ec854x3acf0ce2eb791441@mail.gmail.com>

The "targets" field is used to record the block location(s of the last
allocated block fro a file. It is important to remember this at block
allocation time itself so that data written to the block can be
recovered if the client dies before it finished writing to the entire
block.

Take the case when the client started writing to the 3 datanodes in
the pipeline for that block. Now, let's assume that the client died
aftre writing 100 bytes to the first two replicas whereas the third
replica has only 50 bytes in the block. The namenode holds a lease for
this client. If the client dies prematurely, the namenode experiences
a timeout for the client and triggers lease recovery for the file(s)
that were being written by the client. The namenode has to ensure that
all the three replicas are consistent with one another, otherwise
readers of the block might see inconsistent data depending on which
replica they access. Thus, the namenode has to remember which
datanode(s) possibly has replicas of that block.

hope this help,
dhruba

On Thu, Jul 31, 2008 at 12:27 PM, Sangmin Lee <sangmin.dev@gmail.com> wrote:
> To clarify my question, what is the need for storing the locations of last
> block without being reported by datanodes?
>
> On Thu, Jul 31, 2008 at 3:18 PM, Sangmin Lee <sangmin.dev@gmail.com> wrote:
>
>> Hi folks,
>>
>> I wonder why hadoop.dfs.INodeFileUnderConstruction Class has "targets"
>> field.
>> Can anyone explain this to me?
>> any help would be appreciated.
>>
>> -sangmin
>>
>