hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1186) 0.20: DNs should interrupt writers at start of recovery
Date Thu, 24 Jun 2010 22:38:52 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882376#action_12882376

Todd Lipcon commented on HDFS-1186:

I'm wondering if there is still some sort of weird issue, though...
Let's say there are two concurrent recoveries, one trying to update to genstamp 2 and the
other trying to update to genstamp 3, and there are 3 DNs.

Let's say that GS=2 recovery wins on dn A and B, and GS=3 recovery wins on DN C, a little
bit later. The commitBlockSynchronization() call for GS=3 works, even though the client started
writing again to GS=2.

It's almost as if we need to track through the lease holder name through the block synchronization,
and only allow nextGenerationStamp and commitBlockSynchronization to succeed if the lease
holder agrees?

> 0.20: DNs should interrupt writers at start of recovery
> -------------------------------------------------------
>                 Key: HDFS-1186
>                 URL: https://issues.apache.org/jira/browse/HDFS-1186
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node
>    Affects Versions: 0.20-append
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: hdfs-1186.txt
> When block recovery starts (eg due to NN recovering lease) it needs to interrupt any
writers currently writing to those blocks. Otherwise, an old writer (who hasn't realized he
lost his lease) can continue to write+sync to the blocks, and thus recovery ends up truncating
data that has been sync()ed.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message