hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "alan wootton (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-197) Hook InetAddress.getLocalHost().getHostName() to support cluster simulatation.
Date Fri, 05 May 2006 18:11:29 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-197?page=comments#action_12378087 ] 

alan wootton commented on HADOOP-197:
-------------------------------------


For the purposes of debugging, and generally exploring the behaviour of the distributed hadoop
system, I have constructed a simulation that runs several TaskTrackers, and DataNodes from
Eclipse. In many threads.

This is not a hard trick (and it's very useful, imho) but the DataNodes, for example, need
to think they are all running on different hosts. To do this I define localhost0, localhost1,
etc, just like localhost, and as I launch the DataNodes I give them each their own config
with their own values for a hostname (inetaddress.localhost.name) and also unique values for
dfs.name.dir, dfs.data.dir, etc.

Also, in general, I will wave my arms in the air and say that a single machine might have
any of several names and just using the one from InetAddress may not be the right thing all
the time, for everyone. Or, perhaps nobody cares :-)

While I'm here. Is ClusterTestDFS.java being maintained? It seems broken, and I can't see
how any replication would be allowed, or how this test can barely even run. Put in this patch
and I'll fix it for you.


> Hook InetAddress.getLocalHost().getHostName() to support cluster simulatation.
> ------------------------------------------------------------------------------
>
>          Key: HADOOP-197
>          URL: http://issues.apache.org/jira/browse/HADOOP-197
>      Project: Hadoop
>         Type: Improvement

>   Components: mapred, dfs
>  Environment: all
>     Reporter: alan wootton
>     Priority: Minor
>  Attachments: InetAddressWrapper.patch
>
> I have been running a simulation for weeks now (and also a 30 machine crawl). To make
it work I need to sometimes 
> let DataNodes, and TaskTrackers think they have a different machine-name than the one
in InetAddress.getLocalHost()
> The patch is:
> 1) replace InetAddress.getLocalHost().getHostName() with xxxx
> 	1.a)xxxx could be "conf.get("inetaddress.localhost.name",InetAddress.getLocalHost().getHostName())"
> 	1.b)or, xxxx could be a static call, I chose the latter: "InetAddressWrapper.getLocalHostName(conf)"
> 2) InetAddressWrapper.getLocalHostName(conf) checks the config for a hostname, and then
calls InetAddress.getLocalHost().getHostName()
> There's 3 places where it happens:
> DataNode
> DFSClient
> TaskTracker
> I did not patch the two tests that call InetAddress because they are not really using
hostname.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message