hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bob Hansen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
Date Thu, 10 Dec 2015 16:19:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051168#comment-15051168
] 

Bob Hansen commented on HDFS-9538:
----------------------------------

Proposed design: 
* Add methods to LoadFromFile and OverlayResourceFile to Configuration and HdfsConfiguration
* Have the concept of a search path to separate the concerns of what data is needed (e.g.
core-site.xml) vs. where to find it (e.g. HADOOP_CONF_DIR)
* Doing environmental substitution (HADOOP_CONF_DIR) will be part of HDFS-9385
* This implementation will look for files in the search paths, but will not search inside
of jar files.  While the Java implementation _can_ distribute config files in Jar files, is
that commonly done in the field?
* We will handle implementing the default configuration values in HDFS-9539

> libhdfs++: load configuration from files
> ----------------------------------------
>
>                 Key: HDFS-9538
>                 URL: https://issues.apache.org/jira/browse/HDFS-9538
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>            Reporter: Bob Hansen
>            Assignee: Bob Hansen
>
> One goal of the Configuration classes are to allow the consumers of the libhdfs++ library
to deploy client applications into hadoop edge nodes and have them pick up the Hadoop configuration
that has been deployed there.
> Note that we also need to support the use case where the consumer application will manage
Hadoop configuration files itself, or will handle all configuration out-of-band.
> libhdfs++ should be able to read files that are found in the field and easily construct
an instance that will communicate with the cluster.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message