hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3263) HttpFS should read HDFS config from Hadoop site.xml files
Date Tue, 17 Apr 2012 15:28:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13255643#comment-13255643
] 

Eli Collins commented on HDFS-3263:
-----------------------------------

FileSystemAccessService.java:
- "hdfs.configuration.dir" should be something like "hadoop.conf.dir", as there's nothing
hdfs specific here right?
- Can you re-use an existing mechanism, eg use HADOOP_CONF_DIR by default and only require
the configuration if that's not present?
- Are core-site.xml and hdfs-site.xml the only files that should be pulled in, eg how about
hadoop-policy.xml?
- Rather than  FILE_SYSTEM_SERVICE_CREATED why not use 
- Let's replace NAME_NODE_PROPERTY with FS_DEFAULT_NAME_KEY while we're at it, there shouldn't
be any HDFS-specific names in FileSystemAccesService.java
- Do we need the FILE_SYSTEM_SERVICE_CREATED check? Ie if the proxy is setup correctly can
users ever end up with a conf that doesn't have the hadoop conf set. If so should be a one-time
initialization level check then a check performed on each call to execute.
- If a user gets an HO4 error it means "hdfs.configuration.dir" was not defined, so worth
including that in the message. 
TestHttpFSServer:
- Why use "dummy" for fs.default.name? Don't we need to test this pointing to a real NN?
ServerSetup.apt.vm:
- Prose should say Hadoop configuration files (vs HDFS configuration files)
- On a related node is there a test that covers httpfs working w/ a non-HDFS Hadoop filesystem?
Would be good to have a test that uses httpfs with local file system to catch the introduction
of HDFS-isms. Can punt this to another jira.
                
> HttpFS should read HDFS config from Hadoop site.xml files
> ---------------------------------------------------------
>
>                 Key: HDFS-3263
>                 URL: https://issues.apache.org/jira/browse/HDFS-3263
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.0.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>             Fix For: 2.0.0
>
>         Attachments: HDFS-3263.patch, HDFS-3263.patch, HDFS-3263.patch, HDFS-3263.patch
>
>
> Currently HttpFS reads HDFS client configuration from the httfs-site.xml from any property
of the form 'httpfs.hadoop.conf:HADOOP_PROPERTY'
> This is a bit inconvenient.
> Instead we should support a single property 'httpfs.hadoop.configuration.dir' that can
be pointed to HADOOP conf/ dir and the core-site.xml and hdfs-site.xml files would be read
from there.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message