hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sam rash (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1186) 0.20: DNs should interrupt writers at start of recovery
Date Thu, 24 Jun 2010 23:01:52 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882389#action_12882389
] 

sam rash commented on HDFS-1186:
--------------------------------

I think you can make this argument:

1. each node has to make a transition from x -> x+k
2. at most one node owns any x -> x+k transition as the primary of a recovery
3. success requires all DNs to complete x -> x+k
4. primary then commits x -> x+k

and until commitBlockSync completes, no transition y -> y+j with y > x can come in

right?


> 0.20: DNs should interrupt writers at start of recovery
> -------------------------------------------------------
>
>                 Key: HDFS-1186
>                 URL: https://issues.apache.org/jira/browse/HDFS-1186
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node
>    Affects Versions: 0.20-append
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: hdfs-1186.txt
>
>
> When block recovery starts (eg due to NN recovering lease) it needs to interrupt any
writers currently writing to those blocks. Otherwise, an old writer (who hasn't realized he
lost his lease) can continue to write+sync to the blocks, and thus recovery ends up truncating
data that has been sync()ed.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message