hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-8232) Provide a command line entry point to view/test topology options
Date Fri, 30 Mar 2012 17:14:31 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13242555#comment-13242555
] 

Steve Loughran commented on HADOOP-8232:
----------------------------------------

Test output when a configuration is loaded that lists a map class that does not load
{code}
bash hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/bin/hadoop --config /Users/slo/Java/Hadoop/github/hadoop-patches/misc/topology/invalid-mapper
org.apache.hadoop.net.TopologyTool testfile /Users/slo/Java/Hadoop/github/hadoop-patches/misc/topology/hosts.txt
net.topology.node.switch.mapping.impl=org.apache.hadoop.net.ThisMappingClassIsNotDefined
2012-03-30 18:12:56,225 ERROR net.TopologyTool (TopologyTool.java:run(77)) - Failed to load
the DNS mapping org.apache.hadoop.net.ThisMappingClassIsNotDefined: java.lang.RuntimeException:
java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.net.ThisMappingClassIsNotDefined
not found
java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException:
Class org.apache.hadoop.net.ThisMappingClassIsNotDefined not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1298)
	at org.apache.hadoop.net.AbstractDNSToSwitchMapping.createCachingDNSToSwitchMapping(AbstractDNSToSwitchMapping.java:166)
	at org.apache.hadoop.net.TopologyTool.run(TopologyTool.java:73)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
	at org.apache.hadoop.net.TopologyTool.main(TopologyTool.java:228)
Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.net.ThisMappingClassIsNotDefined
not found
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1266)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1290)
	... 5 more
Caused by: java.lang.ClassNotFoundException: Class org.apache.hadoop.net.ThisMappingClassIsNotDefined
not found
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1175)
	at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1264)
	... 6 more
2012-03-30 18:12:56,229 ERROR net.TopologyTool (TopologyTool.java:run(79)) - The configuration
option is wrong, or the classpath is incomplete
{code}

The error message attempts to hint at the likely causes.
                
> Provide a command line entry point to view/test topology options
> ----------------------------------------------------------------
>
>                 Key: HADOOP-8232
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8232
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: util
>    Affects Versions: 0.23.1
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Minor
>         Attachments: HADOOP-8232.patch
>
>
> Add a new command line entry point "topo" with commands for preflight checking of a clusters
topology setup. 
> The initial operations would be to list the implementation class of the mapper, and attempt
to load it, resolve a set of supplied hostnames, then dump the topology map after the resolution
process.
> Target audience: 
> # ops teams trying to get a new/changed script working before deploying it on a cluster.
> # someone trying to write their first script.
> Resolve and list the rack mappings of the given host
> {code}
> hadoop topo test [host1] [host2] ... 
> {code}
> This would load the hostnames from a given file, resolve all of them and list the results:
> {code}
> hadoop topo testfile filename
> {code}
>  This version is intended for the ops team who have a list of hostnames, IP addresses.

> * Rather than just list them, the ops team may want to mandate that there were no /default-rack
mappings found, as that is invariably a sign that the script isn't handling a hostname properly.
> * No attempt to be clever and do IP address resolution, FQDN to hostname mapping, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message