hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5134) FSNamesystem#commitBlockSynchronization adds under-construction block locations to blocksMap
Date Fri, 13 Feb 2009 22:34:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12673404#action_12673404

Hairong Kuang commented on HADOOP-5134:

> I also added a check that is triggered during lease recovery that matches the crc file
with the data file (in length) before finalizing a block.
Checking the length seems not good enough. We should verify checksum for the last chunk in
the block. Let's open a separate jira for this problem. Also in the code
 if (f.length() > maxDataSize || f.length() < minDataSize)  should be if (f.length()
> maxDataSize || f.length() <= minDataSize).

> FSNamesystem#commitBlockSynchronization adds under-construction block locations to blocksMap
> --------------------------------------------------------------------------------------------
>                 Key: HADOOP-5134
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5134
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.18.2
>            Reporter: Hairong Kuang
>            Assignee: dhruba borthakur
>            Priority: Blocker
>             Fix For: 0.18.4, 0.19.1
>         Attachments: commitBlockSync.patch, commitBlockSync2.patch
> From my understanding of sync/append design, an under construction block should not have
any block locations associated with it in the blocksMap. So an under construction block will
not be managed by ReplicationMonitor.
> However, if there is an error in the write pipeline, a lease recovery will trigger a
call, commitBlockSynchronization, to NN. This call will add the successfully-recovered datanodes
to blocksMap. This seems to violate the design. It should update the targets of the last block
at INode instead. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message