hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "alan wootton (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-197) Hook InetAddress.getLocalHost().getHostName() to support cluster simulatation.
Date Fri, 05 May 2006 18:11:29 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-197?page=comments#action_12378087 ] 

alan wootton commented on HADOOP-197:

For the purposes of debugging, and generally exploring the behaviour of the distributed hadoop
system, I have constructed a simulation that runs several TaskTrackers, and DataNodes from
Eclipse. In many threads.

This is not a hard trick (and it's very useful, imho) but the DataNodes, for example, need
to think they are all running on different hosts. To do this I define localhost0, localhost1,
etc, just like localhost, and as I launch the DataNodes I give them each their own config
with their own values for a hostname (inetaddress.localhost.name) and also unique values for
dfs.name.dir, dfs.data.dir, etc.

Also, in general, I will wave my arms in the air and say that a single machine might have
any of several names and just using the one from InetAddress may not be the right thing all
the time, for everyone. Or, perhaps nobody cares :-)

While I'm here. Is ClusterTestDFS.java being maintained? It seems broken, and I can't see
how any replication would be allowed, or how this test can barely even run. Put in this patch
and I'll fix it for you.

> Hook InetAddress.getLocalHost().getHostName() to support cluster simulatation.
> ------------------------------------------------------------------------------
>          Key: HADOOP-197
>          URL: http://issues.apache.org/jira/browse/HADOOP-197
>      Project: Hadoop
>         Type: Improvement

>   Components: mapred, dfs
>  Environment: all
>     Reporter: alan wootton
>     Priority: Minor
>  Attachments: InetAddressWrapper.patch
> I have been running a simulation for weeks now (and also a 30 machine crawl). To make
it work I need to sometimes 
> let DataNodes, and TaskTrackers think they have a different machine-name than the one
in InetAddress.getLocalHost()
> The patch is:
> 1) replace InetAddress.getLocalHost().getHostName() with xxxx
> 	1.a)xxxx could be "conf.get("inetaddress.localhost.name",InetAddress.getLocalHost().getHostName())"
> 	1.b)or, xxxx could be a static call, I chose the latter: "InetAddressWrapper.getLocalHostName(conf)"
> 2) InetAddressWrapper.getLocalHostName(conf) checks the config for a hostname, and then
calls InetAddress.getLocalHost().getHostName()
> There's 3 places where it happens:
> DataNode
> DFSClient
> TaskTracker
> I did not patch the two tests that call InetAddress because they are not really using

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

View raw message