hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10610) DfsClient doesn't add hdfs-site.xml as a resource
Date Tue, 26 Jul 2016 08:43:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15393441#comment-15393441
] 

Steve Loughran commented on HDFS-10610:
---------------------------------------

I think my stance on HdfsConfiguration is well documented: whether the HDFS team choose to
accept it or not, it's a defacto part of the public API, its constants are imported downstream,
hdfs-site is expected to be loaded —and if you change things without asking people downstream
are unhappy. HDFS-9301 spells out my concerns in more detail.

+1 for for making it public. 

In YARN-679, my generic service launcher, I explicitly load in the hdfs-site and yarn-site
XML files early, plus you can specify extra resources (hbase-site.xml) to also load. This
gets everything pulled in before you go anywhere near a filesystem API *or UGI*.

> DfsClient doesn't add hdfs-site.xml as a resource
> -------------------------------------------------
>
>                 Key: HDFS-10610
>                 URL: https://issues.apache.org/jira/browse/HDFS-10610
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 2.8.0
>            Reporter: Eric Badger
>
> Instantiating a new DfsClient used to add hdfs-site.xml as a resource in 2.7, but that
compatibility has been broken in 2.8. This only accidentally worked in 2.7, since DfsClient
would load HdfsConstants, which would in turn create the static IO_FILE_BUFFER_SIZE, which
would instantiate an HdfsConfiguration, which would finally add hdfs-site.xml. 
> In 2.8, IO_FILE_BUFFER_SIZE no longer exists in HdfsConstants.java and so this no longer
works out of coincidence. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message