Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Fri, 30 Mar 2012 17:24:34 +0000 (UTC)
From: "Steve Loughran (Commented) (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: 
 <250322388.38834.1333128274226.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1006003888.38571.1333125391641.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HADOOP-8232) Provide a command line entry point
 to view/test topology options
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242565#comment-13242565 ] 

Steve Loughran commented on HADOOP-8232:
----------------------------------------

Test with table mapping pointing to a table that does not exist relative to the directory where the hadoop command is being invoked from:
{code}
bash hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/bin/hadoop --config /Users/slo/Java/Hadoop/github/hadoop-patches/misc/topology/missing-table org.apache.hadoop.net.TopologyTool test 4 home extra 
net.topology.node.switch.mapping.impl=org.apache.hadoop.net.TableMapping
Caching wrapper class = class org.apache.hadoop.net.TableMapping
Instance information: cached switch mapping relaying to TableMapping with table "table.txt"
Path: /Users/slo/Java/Hadoop/github/trunk/table.txt
Map size: 0
Topology is not  considered single-switch
Resolving 4
2012-03-30 18:22:03,831 WARN  net.TableMapping (TableMapping.java:load(113)) - table.txt cannot be read. /default-rack will be returned.
java.io.FileNotFoundException: table.txt (No such file or directory)
	at java.io.FileInputStream.open(Native Method)
	at java.io.FileInputStream.<init>(FileInputStream.java:120)
	at java.io.FileInputStream.<init>(FileInputStream.java:79)
	at java.io.FileReader.<init>(FileReader.java:41)
	at org.apache.hadoop.net.TableMapping$RawTableMapping.load(TableMapping.java:98)
	at org.apache.hadoop.net.TableMapping$RawTableMapping.resolve(TableMapping.java:132)
	at org.apache.hadoop.net.CachedDNSToSwitchMapping.resolve(CachedDNSToSwitchMapping.java:119)
	at org.apache.hadoop.net.TopologyTool.resolveHostnames(TopologyTool.java:189)
	at org.apache.hadoop.net.TopologyTool.run(TopologyTool.java:103)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
	at org.apache.hadoop.net.TopologyTool.main(TopologyTool.java:228)
Hostname "4" resolved to "/default-rack"  in 15.548 milliseconds
Resolving home
Hostname "home" resolved to "/default-rack"  in 6.812 milliseconds
Resolving extra
Hostname "extra" resolved to "/default-rack"  in 0.702 milliseconds

Final topology:

Mapping: cached switch mapping relaying to TableMapping with table "table.txt"
Path: /Users/slo/Java/Hadoop/github/trunk/table.txt
Map size: 0
Known mappings:
  home -> /default-rack
  extra -> /default-rack
  4 -> /default-rack
Nodes: 3
Switches: 1

{code}
The error message is printed inside the TableMapping, and is not visible to the caller. It could be improved by listing the absolute path to the table.
                
> Provide a command line entry point to view/test topology options
> ----------------------------------------------------------------
>
>                 Key: HADOOP-8232
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8232
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: util
>    Affects Versions: 0.23.1
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Minor
>         Attachments: HADOOP-8232.patch
>
>
> Add a new command line entry point "topo" with commands for preflight checking of a clusters topology setup. 
> The initial operations would be to list the implementation class of the mapper, and attempt to load it, resolve a set of supplied hostnames, then dump the topology map after the resolution process.
> Target audience: 
> # ops teams trying to get a new/changed script working before deploying it on a cluster.
> # someone trying to write their first script.
> Resolve and list the rack mappings of the given host
> {code}
> hadoop topo test [host1] [host2] ... 
> {code}
> This would load the hostnames from a given file, resolve all of them and list the results:
> {code}
> hadoop topo testfile filename
> {code}
>  This version is intended for the ops team who have a list of hostnames, IP addresses. 
> * Rather than just list them, the ops team may want to mandate that there were no /default-rack mappings found, as that is invariably a sign that the script isn't handling a hostname properly.
> * No attempt to be clever and do IP address resolution, FQDN to hostname mapping, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira