hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HDFS-5988) Bad fsimage generated after upgrade
Date Thu, 20 Feb 2014 23:05:25 GMT

     [ https://issues.apache.org/jira/browse/HDFS-5988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Wang updated HDFS-5988:
------------------------------

    Attachment: hdfs-5988-1.patch

{{FSImageFormat#Loader}} was incorrectly basing the decision to populate the {{FSDirectory#inodeMap}}
based on if the old fsimage layout version supported inodes. We only see the error with the
new PB-based image, since it iterates through {{inodeMap}}, while the old fsimage saver would
traverse the directory structure instead.

I also added a bunch of trace/debug logging to OIV, which was helpful in tracking down this
issue. Trust me, lot of effort for a one-line fix :)

> Bad fsimage generated after upgrade
> -----------------------------------
>
>                 Key: HDFS-5988
>                 URL: https://issues.apache.org/jira/browse/HDFS-5988
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: 2.4.0
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>            Priority: Blocker
>         Attachments: hdfs-5988-1.patch
>
>
> Internal testing revealed an issue where, after upgrading from an earlier release, we
always fail to save a correct PB-based fsimage (namely, missing inodes leading to an inconsistent
namespace). This results in substantial data loss, since the upgraded fsimage is broken, as
well as the fsimages generated by saveNamespace and checkpointing.
> This ended up being a bug in the old fsimage loading code, patch coming.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message