hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-1505) saveNamespace appears to succeed even if all directories fail to save
Date Thu, 12 May 2011 20:20:47 GMT

    [ https://issues.apache.org/jira/browse/HDFS-1505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13032641#comment-13032641

Aaron T. Myers commented on HDFS-1505:

Thanks a lot for the review/comments, Matt.

Upon further reflection, I think the desired check should actually be:

    if ((storage.getNumStorageDirs(NameNodeDirType.IMAGE) == 0 ||
        storage.getNumStorageDirs(NameNodeDirType.EDITS) == 0) &&
        storage.getNumStorageDirs(NameNodeDirType.IMAGE_AND_EDITS) == 0) {
      throw new IOException("Failed to save any storage directories while saving namespace");

What do you think? Note that IMAGE_AND_EDITS is a distinct type of storage directory, which
contains both {{fsimage}} and {{edits}} files. Apologies if you already knew that.

bq. Also, since HDFS-1826 copied the concurrent saveNamespace() logic into FSImage.doUpgrade(),
would you please add the same code fragment to the end of doUpgrade(), and a corresponding
corruption unit test case to TestDFSUpgrade? Thanks.

Will do.

bq. I would suggest fixing the lack of notification in FSEditLog.open(), but also in your
patch to saveNamespace() the check for empty IMAGE and EDITS lists should precede the call
to editLog.open().

Agreed. Will do.

> saveNamespace appears to succeed even if all directories fail to save
> ---------------------------------------------------------------------
>                 Key: HDFS-1505
>                 URL: https://issues.apache.org/jira/browse/HDFS-1505
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 0.22.0, 0.23.0
>            Reporter: Todd Lipcon
>            Assignee: Aaron T. Myers
>            Priority: Blocker
>             Fix For: 0.22.0
>         Attachments: hdfs-1505-22.0.patch, hdfs-1505-22.1.patch, hdfs-1505-test.txt,
hdfs-1505-trunk.0.patch, hdfs-1505-trunk.1.patch
> After HDFS-1071, saveNamespace now appears to "succeed" even if all of the individual
directories failed to save.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message