hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5709) Improve upgrade with existing files and directories named ".snapshot"
Date Sat, 04 Jan 2014 00:08:51 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13862066#comment-13862066

Andrew Wang commented on HDFS-5709:

These are good points. I agree that we shouldn't do a silent rename, and a mandatory pre-upgrade
tool isn't a great option either. However, making the user ls -R and rename everything is
still a pretty big hassle. After the rename, I think you'd also need to saveNamespace on both
NNs to flush any references to .snapshot in the edit log.

Maybe we could have a config option (by default unset) that will rename a reserved path on
upgrade to a user-provided one. Something like {{dfs.namenode.upgrade.rename.reserved.snapshot=.user-snapshot}}.
If we need more complex behavior, we could even do something with path globbing.

This also might not be the last time we want to add a new reserved path, so it seems worth
having a better upgrade story for the future.

> Improve upgrade with existing files and directories named ".snapshot"
> ---------------------------------------------------------------------
>                 Key: HDFS-5709
>                 URL: https://issues.apache.org/jira/browse/HDFS-5709
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 3.0.0, 2.2.0
>            Reporter: Andrew Wang
>              Labels: snapshots, upgrade
> Right now in trunk, upgrade fails messily if the old fsimage or edits refer to a directory
named ".snapshot". We should at least print a better error message (which I believe was the
original intention in HDFS-4666), and [~atm] proposed automatically renaming these files and

This message was sent by Atlassian JIRA

View raw message