hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uma Maheswara Rao G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4516) Client crash after block allocation and NN switch before lease recovery for the same file can cause readers to fail forever
Date Sat, 23 Feb 2013 05:58:13 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585026#comment-13585026
] 

Uma Maheswara Rao G commented on HDFS-4516:
-------------------------------------------

One more option what I can see is, How about not persisting block along with allocate block.
and persist the block with separate call to NN after createBlockOutPutStream success.

If createBlockOutputStream success, block must have create in DNs, So now that is correct
time to persist in NN also before start writing any content i.e, just after createBlockOutputStream
success.
But I know, this will introduce one extra NN call for block and will have performnace impact.
Somewhat may compensated by not persisting the blocks along with allocate block but still
have NW call. Any other thoughts/suggestions?
                
> Client crash after block allocation and NN switch before lease recovery for the same
file can cause readers to fail forever
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-4516
>                 URL: https://issues.apache.org/jira/browse/HDFS-4516
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 3.0.0, 2.0.3-alpha
>            Reporter: Uma Maheswara Rao G
>            Priority: Critical
>
> If client crashes just after allocating block( blocks not yet created in DNs) and NN
also switched after this, then new Namenode will not know about locs.
> Further details will be in comment.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message