hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bob Hansen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9538) libhdfs++: load configuration from files
Date Thu, 10 Dec 2015 16:19:11 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9538?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15051168#comment-15051168

Bob Hansen commented on HDFS-9538:

Proposed design: 
* Add methods to LoadFromFile and OverlayResourceFile to Configuration and HdfsConfiguration
* Have the concept of a search path to separate the concerns of what data is needed (e.g.
core-site.xml) vs. where to find it (e.g. HADOOP_CONF_DIR)
* Doing environmental substitution (HADOOP_CONF_DIR) will be part of HDFS-9385
* This implementation will look for files in the search paths, but will not search inside
of jar files.  While the Java implementation _can_ distribute config files in Jar files, is
that commonly done in the field?
* We will handle implementing the default configuration values in HDFS-9539

> libhdfs++: load configuration from files
> ----------------------------------------
>                 Key: HDFS-9538
>                 URL: https://issues.apache.org/jira/browse/HDFS-9538
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>            Reporter: Bob Hansen
>            Assignee: Bob Hansen
> One goal of the Configuration classes are to allow the consumers of the libhdfs++ library
to deploy client applications into hadoop edge nodes and have them pick up the Hadoop configuration
that has been deployed there.
> Note that we also need to support the use case where the consumer application will manage
Hadoop configuration files itself, or will handle all configuration out-of-band.
> libhdfs++ should be able to read files that are found in the field and easily construct
an instance that will communicate with the cluster.

This message was sent by Atlassian JIRA

View raw message