hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba borthakur (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-5134) FSNamesystem#commitBlockSynchronization adds under-construction block locations to blocksMap
Date Wed, 11 Feb 2009 08:03:00 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

dhruba borthakur updated HADOOP-5134:
-------------------------------------

    Attachment: commitBlockSync.patch

This patch does the following:

1. A commitBlockSync does not add block locations of the last block to the blocksMap, it just
stores it in the INodeUnderConstruction. However, if the commitBlockSync occurs as part of
lease recovery that is triggered by the NN (i.e., the file is getting closed), then it does
update the blocksMap.
2. Opening a file for append causes the last partial block of the file (if any) to move its
block locations from the blocksMap to the INodeUnderConstruction.

> FSNamesystem#commitBlockSynchronization adds under-construction block locations to blocksMap
> --------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-5134
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5134
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.18.2
>            Reporter: Hairong Kuang
>            Assignee: dhruba borthakur
>            Priority: Blocker
>             Fix For: 0.18.4
>
>         Attachments: commitBlockSync.patch
>
>
> From my understanding of sync/append design, an under construction block should not have
any block locations associated with it in the blocksMap. So an under construction block will
not be managed by ReplicationMonitor.
> However, if there is an error in the write pipeline, a lease recovery will trigger a
call, commitBlockSynchronization, to NN. This call will add the successfully-recovered datanodes
to blocksMap. This seems to violate the design. It should update the targets of the last block
at INode instead. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message