hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sam rash (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-1186) 0.20: DNs should interrupt writers at start of recovery
Date Thu, 24 Jun 2010 21:54:51 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1186?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882350#action_12882350

sam rash commented on HDFS-1186:

hey todd,

i was looking at this patch, and while it has certainly reduced the chance of problems, isn't
it still possible a new writer thread could be created 

1. between the kill loop in startBlockRecovery() and the synchronized block
2. between the startBlockRecovery() call and updateBlock() call

I seem to recall reasoning with dhruba that while in theory these could occur from the DN
perspective, the circumstances that would have to occur outside were not (once you fixed hdfs-1260
anyway, where genstamp checks work right in concurrent lease recovery).

what's your take on this?  is it full-proof now?  (1 & 2 can't happen)  or what about
introducing a state like RUR here?  (at least disabling writes to a block while under recovery,
maybe timing out in case the lease recovery owner dies)

> 0.20: DNs should interrupt writers at start of recovery
> -------------------------------------------------------
>                 Key: HDFS-1186
>                 URL: https://issues.apache.org/jira/browse/HDFS-1186
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: data-node
>    Affects Versions: 0.20-append
>            Reporter: Todd Lipcon
>            Assignee: Todd Lipcon
>            Priority: Blocker
>         Attachments: hdfs-1186.txt
> When block recovery starts (eg due to NN recovering lease) it needs to interrupt any
writers currently writing to those blocks. Otherwise, an old writer (who hasn't realized he
lost his lease) can continue to write+sync to the blocks, and thus recovery ends up truncating
data that has been sync()ed.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message