hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1218) 20 append: Blocks recovered on startup should be treated with lower priority during block synchronization
Date Tue, 22 Jun 2010 01:45:57 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881007#action_12881007
] 

Todd Lipcon commented on HDFS-1218:
-----------------------------------

re #1: I was referring to client-initiated recovery after a soft lease expiration (like what
we do in all of the TestFileAppend4 tests). In that case, the targets for recovery come from
the "targets" field of the last block, which will include the previously-down node, which
may well have restarted by this time.

Regarding simplifying code, perhaps... but I think it's a bit late in the game to change it
much now :)

re #2. I think that's fine - we'll use a shorter length, but in this case the client never
received ACK for those extra bytes. Thus, no sync() could have succeeded, and it's OK to truncate
those extra bytes, even though we happen to have them.

> 20 append: Blocks recovered on startup should be treated with lower priority during block
synchronization
> ---------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-1218
>                 URL: https://issues.apache.org/jira/browse/HDFS-1218
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node
>    Affects Versions: 0.20-append
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Critical
>             Fix For: 0.20-append
>
>         Attachments: hdfs-1281.txt
>
>
> When a datanode experiences power loss, it can come back up with truncated replicas (due
to local FS journal replay). Those replicas should not be allowed to truncate the block during
block synchronization if there are other replicas from DNs that have _not_ restarted.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message