hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Thomas <tho...@hep.caltech.edu>
Subject Re: names or ips in rack awareness script?
Date Thu, 19 Nov 2009 16:18:19 GMT
Steve Loughran wrote:
> Michael Thomas wrote:
>> IPs are passed to the rack awareness script.  We use 'dig' to do the
>> reverse lookup to find the hostname, as we also embed the rack id in
>> the worker node hostnames.
>>
> 
> It might be nice to have some example scripts up on the wiki, to give
> people a good starting place

If somebody with write access to the wiki would like to add it, here is
the one we use on our Rocks cluster.

--Mike


#!/bin/sh

# The default rule assumes that the nodes are connected to the PDU
# and switch located in the same rack.  Only the exceptions need
# to be listed here.

# In our Rocks cluster, nodes are named "compute-X-Y", where X is the
# Rack identifier and Y is the vertical position of the node within
# the rack.

for ip in $@ ; do
    hostname=`nslookup $ip | grep "name =" | awk '{print $4}' | sed -e
's/\.local\.$//' `
    case $hostname in
        compute-5-8)
            # Exception: This node had to be rewired into
            # an adjacent rack
            rack="/Rack4"
            ;;
        *)
            rack=`echo $hostname | sed -e
's/^[a-z]*-\([0-9]*\)-[0-9]*.*/\/Rack\1/'`
            ;;
        esac
        echo $rack
done

Mime
View raw message