hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bob Hansen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9117) Config file reader / options classes for libhdfs++
Date Wed, 28 Oct 2015 17:30:27 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14978833#comment-14978833

Bob Hansen commented on HDFS-9117:

The current design has the core of the C++ engine (FileSystem, InputStream, etc.) dependent
solely on the Options class, as we would all like.   I believe that the Configuration class
should be part of the larger C++ API, however, since it is a requirement for interoperability
with deployed installations.

I propose we leave the base Configuration class in lib/common, and move the hdfs_configuration
class to lib/config.  The HDFSConfiguration class will continue to be responsible for mapping
keys to hdfs::Option members and producing the hdfs::Option object that the FileSystem and
friends will consume.  Does that seem an appropriate approach?

bq. Environment variables and substitutions are clearly platform-dependent.
getenv is part of the C standard library, and the substitutions are entirely the purview of
the Java and C++ Configuration implementation, so I don't see how they are at all platform-dependent.
 We can wrap them in configuration #ifdefs to support platforms that for some reason don't
support the C standard library, but I don't see that as being a great service to our consumer

The configuration tests use setenv, which is available on all unix-y platforms.  While I hope
this library will eventually support Windows clients, we don't currently build or test on
that platform, and I expect the setenv/_putenv refactoring will be the smallest part of the
work.  Again, to guard against that, we can #ifdef out those tests based on the available

> Config file reader / options classes for libhdfs++
> --------------------------------------------------
>                 Key: HDFS-9117
>                 URL: https://issues.apache.org/jira/browse/HDFS-9117
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs-client
>    Affects Versions: HDFS-8707
>            Reporter: Bob Hansen
>            Assignee: Bob Hansen
>         Attachments: HDFS-9117.HDFS-8707.001.patch, HDFS-9117.HDFS-8707.002.patch, HDFS-9117.HDFS-8707.003.patch,
HDFS-9117.HDFS-8707.004.patch, HDFS-9117.HDFS-8707.005.patch, HDFS-9117.HDFS-8707.006.patch,
HDFS-9117.HDFS-8707.008.patch, HDFS-9117.HDFS-8707.009.patch, HDFS-9117.HDFS-9288.007.patch
> For environmental compatability with HDFS installations, libhdfs++ should be able to
read the configurations from Hadoop XML files and behave in line with the Java implementation.
> Most notably, machine names and ports should be readable from Hadoop XML configuration
> Similarly, an internal Options architecture for libhdfs++ should be developed to efficiently
transport the configuration information within the system.

This message was sent by Atlassian JIRA

View raw message