hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5709) Improve upgrade with existing files and directories named ".snapshot"
Date Fri, 24 Jan 2014 19:24:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13881336#comment-13881336

Andrew Wang commented on HDFS-5709:

I'm not sure if this adds anything to the discussion, but the current workflow in the patch

* By default, the config option is not set
* On upgrade, if we hit a .snapshot path, we throw an exception and abort telling the user
to set the config option
* When the config option is set, there are log prints whenever something is renamed

This feels procedurally pretty similar to the alternative of a special startup option, and
it's only required to be set if we hit a .snapshot path. Most users will never need to worry
about this config.

I agree that doing renames with the context of the full parent path (e.g. /X/.snapshot renames
to /X/.myx-snapshot, /Y/.snapshot renames to /Y/.myy.snapshot) would be more powerful, but
IIUC that implies scanning the fsimage and logs initially to enumerate all the .snapshot paths,
have the user configure their rename rules, then load the fsimage/logs again to do the rename.

Suresh, does any of this influence your view? I'd really like to move forward on this issue.

> Improve upgrade with existing files and directories named ".snapshot"
> ---------------------------------------------------------------------
>                 Key: HDFS-5709
>                 URL: https://issues.apache.org/jira/browse/HDFS-5709
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 3.0.0, 2.2.0
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>              Labels: snapshots, upgrade
>         Attachments: hdfs-5709-1.patch, hdfs-5709-2.patch, hdfs-5709-3.patch, hdfs-5709-4.patch,
> Right now in trunk, upgrade fails messily if the old fsimage or edits refer to a directory
named ".snapshot". We should at least print a better error message (which I believe was the
original intention in HDFS-4666), and [~atm] proposed automatically renaming these files and

This message was sent by Atlassian JIRA

View raw message