Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <115568853.1234379939667.JavaMail.jira@brutus>
Date: Wed, 11 Feb 2009 11:18:59 -0800 (PST)
From: "dhruba borthakur (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-5134)
 FSNamesystem#commitBlockSynchronization adds under-construction block
 locations to blocksMap
In-Reply-To: <972388103.1233104219625.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672734#action_12672734 ] 

dhruba borthakur commented on HADOOP-5134:
------------------------------------------

This patch had nothing much to do with "appends", but it has mightily related to datanode(s) dropping off the write-pipeline during a write form a client.

When a client is writing to a file and encounters an error (bad datanode in pipleine), it invokes commitBlockSync(). This patch guarantees that this call to commitBlockSyn() does not add any replicas to the blocksMap. That's what you wanted, isn't it?

> FSNamesystem#commitBlockSynchronization adds under-construction block locations to blocksMap
> --------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5134
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5134
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.18.2
>            Reporter: Hairong Kuang
>            Assignee: dhruba borthakur
>            Priority: Blocker
>             Fix For: 0.18.4
>
>         Attachments: commitBlockSync.patch
>
>
> From my understanding of sync/append design, an under construction block should not have any block locations associated with it in the blocksMap. So an under construction block will not be managed by ReplicationMonitor.
> However, if there is an error in the write pipeline, a lease recovery will trigger a call, commitBlockSynchronization, to NN. This call will add the successfully-recovered datanodes to blocksMap. This seems to violate the design. It should update the targets of the last block at INode instead. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.