hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Suresh Srinivas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5709) Improve upgrade with existing files and directories named ".snapshot"
Date Fri, 03 Jan 2014 19:39:52 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13861835#comment-13861835

Suresh Srinivas commented on HDFS-5709:

bq. Hey Suresh, could you elaborate a bit on your -1?
Sorry I should have added more details in my previous comment. Silently renaming it will cause
issues to any applications that depend on the name .snapshot. I would rather make it an active
decision during upgrade so that the admin can take care of renaming to appropriate name he
chooses and also ensure applications that depend on such naming are updated. Otherwise upgrade
will silently succeed followed by failures.

I am assuming that .snapshot is used in very few clusters. In such a case cost of rollback
and rename probably should not affect large number of clusters.

As regards to tool to run for pre-upgrade, there are several issues:
# This tool can be available only in later releases (as we cannot change older releases already
installed on the cluster).
# A user may not run the tool, even if it is made available on newer release and will have
to rollback (as you pointed out)
# The pre-upgrade tool must consume fsimage and editlog (even if there is no .snapshot file/directory).
This will extend the time for upgrade for every upgrade.

One possible solution is to turn of snapshot feature post upgade and allow .snapshot directory
or file. When a user tries to turn on .snapshot the namenode fails to come up, forcing them
to turn off the feature, rename directories and turn it back on again. This may be a lot of

> Improve upgrade with existing files and directories named ".snapshot"
> ---------------------------------------------------------------------
>                 Key: HDFS-5709
>                 URL: https://issues.apache.org/jira/browse/HDFS-5709
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 3.0.0, 2.2.0
>            Reporter: Andrew Wang
>              Labels: snapshots, upgrade
> Right now in trunk, upgrade fails messily if the old fsimage or edits refer to a directory
named ".snapshot". We should at least print a better error message (which I believe was the
original intention in HDFS-4666), and [~atm] proposed automatically renaming these files and

This message was sent by Atlassian JIRA

View raw message