hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4820) Remove hdfs-default.xml
Date Fri, 17 May 2013 16:55:18 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13660852#comment-13660852

Chris Nauroth commented on HDFS-4820:

Removing *-default.xml seems to complicate resolution of some of our more dynamic configuration
properties.  A good example is our mapping of file system impl classes by URI scheme used
by {{AbstractFileSystem#createFileSystem}}:

  <description>The AbstractFileSystem for file: uris.</description>

  <description>The FileSystem for hdfs: uris.</description>

  public static AbstractFileSystem createFileSystem(URI uri, Configuration conf)
      throws UnsupportedFileSystemException {
    Class<?> clazz = conf.getClass("fs.AbstractFileSystem." + 
                                uri.getScheme() + ".impl", null);
    if (clazz == null) {
      throw new UnsupportedFileSystemException(
          "No AbstractFileSystem for scheme: " + uri.getScheme());
    return (AbstractFileSystem) newInstance(clazz, uri, conf);

Without defaults in the XML, this code will need to hard-code the mapping somewhere.  We'll
have to remember to cover all cases like this.

...it should not be part of the jar and should not be looked for and loaded in by default
into the Configuration object.

This may be a bigger concern for compatibility.  {{Configuration}} is annotated public/stable,
and I've seen a lot of tutorials with sample code that instantiates a new instance and expects
it to be fully populated with the keys from *-default.xml.  For full compatibility, I suppose
we'd need to update not only our own {{Configuration#get}} calls to enforce the defaults,
but also guarantee that if a client creates a new instance, they get the same values that
used to be provided in the XML.  Again, this probably would involve some kind of hard-coding
during static initialization.
> Remove hdfs-default.xml
> -----------------------
>                 Key: HDFS-4820
>                 URL: https://issues.apache.org/jira/browse/HDFS-4820
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.0.4-alpha
>            Reporter: Siddharth Seth
> Similar to YARN-673, which contains additional details.
> There's separate jiras for YARN, MR and HDFS so enough people take a look. Looking for
reasons for these files to exist, other than the ones mentioned in YARN-673, or a good reason
to keep the files.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message