hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aaron T. Myers (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5709) Improve upgrade with existing files and directories named ".snapshot"
Date Tue, 21 Jan 2014 03:35:19 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13877144#comment-13877144
] 

Aaron T. Myers commented on HDFS-5709:
--------------------------------------

Hey guys,

Jing - I'm not in favor of a generic prefix for all reserved names because I don't think it's
flexible enough. A user should be able to rename a reserved component name to a specific other
reserved component name. With just a prefix mutation, I could easily imagine having to run
some operation after the NN had started up to then find all of the renamed paths and again
rename them to some other name with the prefix removed. That's a pain that we shouldn't put
our users through.

Suresh - with regard to the suggestion that we have a completely generic system of renaming
reserved component names, how strongly do you see that as a requirement? I personally view
that as overkill for two reasons: 1) Empirically we rarely add reserved names (only two in
the lifetime of HDFS so far) and I don't anticipate adding many more, and 2) we can't necessarily
anticipate the precise semantics of all reserved names. For example, ".reserved" should only
be considered invalid and renamed if it's in the root directory. I think the solution for
that is not to develop a completely generic system for renaming path components, but rather
to do as Andrew suggests and add a conf option per reserved name, and add the code necessary
for each one as it comes up.

I'm in favor of checking in this patch as-is. If we later decide to add a more generic system,
we can do that and quite easily maintain backward compatibility of this conf option with that
system. I wouldn't personally object to implementing a K/V config setting system as Andrew
suggested, but again, seems like overkill to me.

> Improve upgrade with existing files and directories named ".snapshot"
> ---------------------------------------------------------------------
>
>                 Key: HDFS-5709
>                 URL: https://issues.apache.org/jira/browse/HDFS-5709
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>    Affects Versions: 3.0.0, 2.2.0
>            Reporter: Andrew Wang
>            Assignee: Andrew Wang
>              Labels: snapshots, upgrade
>         Attachments: hdfs-5709-1.patch, hdfs-5709-2.patch, hdfs-5709-3.patch, hdfs-5709-4.patch,
hdfs-5709-5.patch
>
>
> Right now in trunk, upgrade fails messily if the old fsimage or edits refer to a directory
named ".snapshot". We should at least print a better error message (which I believe was the
original intention in HDFS-4666), and [~atm] proposed automatically renaming these files and
directories.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message