hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-3263) HttpFS should read HDFS config from Hadoop site.xml files
Date Wed, 18 Apr 2012 18:28:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-3263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13256786#comment-13256786

Eli Collins commented on HDFS-3263:

bq. TUCU: No we cannot, the current mechanism relies on having the HADOOP_CONF_DIR in the
classpath. HttpFS does not have HADOOP_CONF_DIR in the classpath (furthermore, it can run
in a setup where there is no other hadoop component installed)

I was thinking that it might make sense to keep the mechanism the same so that we don't have
two separate ways to specify the conf dir (a HADOOP_CONF_DIR env dir being the normal way
hadoop clients locate config vs the new "config.dir" parameter that essentially does the same).
I don't feel strongly.

bq. TUCU: was not aware of the hadoop-policy.xml but how would the be used in HttpFS context?

I checked and it looks like all the relevant keys are only used on the server side so I don't
think you need to include it here.

bq. This check is to prevent httpfs code (got bitten by this) to create and use a Configuration
object without the hadoop confs in it (per first bullet item)

Makes sense.

+1 modulo the following nits

Nit: How about renaming "config.dir" to "hadoop.conf.dir" 
Nit: HttpFSServerWebApp line is way over 80 chars now, please re-format
> HttpFS should read HDFS config from Hadoop site.xml files
> ---------------------------------------------------------
>                 Key: HDFS-3263
>                 URL: https://issues.apache.org/jira/browse/HDFS-3263
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>    Affects Versions: 2.0.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>             Fix For: 2.0.0
>         Attachments: HDFS-3263.patch, HDFS-3263.patch, HDFS-3263.patch, HDFS-3263.patch,
> Currently HttpFS reads HDFS client configuration from the httfs-site.xml from any property
of the form 'httpfs.hadoop.conf:HADOOP_PROPERTY'
> This is a bit inconvenient.
> Instead we should support a single property 'httpfs.hadoop.configuration.dir' that can
be pointed to HADOOP conf/ dir and the core-site.xml and hdfs-site.xml files would be read
from there.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message