hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-6610) Hadoop conf/ servlet improvements
Date Tue, 18 Oct 2016 21:45:58 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-6610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Andrew Wang updated HADOOP-6610:
    Fix Version/s:     (was: 2.8.0)

> Hadoop conf/ servlet improvements
> ---------------------------------
>                 Key: HADOOP-6610
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6610
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: conf
>    Affects Versions: 0.22.0
>            Reporter: Steve Loughran
>            Priority: Minor
> I'm playing with the conf/ servlet, trying to do a workflow that
> # pulls down the conf servlet from a well known URL (this is trickier when your VMs are
dynamic, but possible)
> # saves it locally, using {{<get>}} task
> #  {{<get>}} some info on the machines in the allocated cluster, like their external
> # SCP in the configuration files, JAR files needed to submit work, 
> # submit work via SSH
> I have to SSH as the VMs have different internal/external addresses; HDFS gets upset.
> Some issues I've found so far
> # It's good to set expires headers on everything; HADOOP-6607 covers that
> # Having sorted conf values makes it easier to locate properties, otherwise you have
to save it to a text editor and search around
> # the <!-- Loaded from Unknown --> option makes things noisy
> # Saving as a java.util.Properties would let me pull these things into a build file or
other tool very easily. This is easy to test too.
> # Have a comment at the top listing when the conf was generated, and the hostname. Maybe
even make them conf values
> More tricky is the conf options that are dynamic, things like 
> {code}
> <property><!--Loaded from Unknown--><name>dfs.datanode.address</name><value></value></property>
> {code}
> These show what the node was started with, not what it actually got. I am doing a workaround
there with my code (setting the actual values in the conf file with {{live.dfs.datanode.address}},
etc, and extracting them that way. I don't want to lose the original values, but do want the
real ones

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message