hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Rosenstrauch <dar...@darose.net>
Subject Re: Hadoop topology not working (all servers belongs to default rack)
Date Thu, 28 Jun 2012 14:46:54 GMT
On 06/27/2012 11:32 PM, Ben Kim wrote:
> Hi
> I got my topology script from
> http://wiki.apache.org/hadoop/topology_rack_awareness_scripts
> I checked that the script works correctly.
>
> But, in the hadoop cluster, all my servers get assigned to the default rack.
> I'm using hadoop 1.0.3, but had experienced same problem with 1.0.0 version.
>
>
> Yunhong was having the same problem in the past without any resolution.
> http://mail-archives.apache.org/mod_mbox/hadoop-common-user/200807.mbox/%3CPine.LNX.4.64.0807031453070.28079@bert.cs.uic.edu%3E
>
> *Benjamin Kim*
> *benkimkimben at gmail*

We've used this script for rack awareness:

#!/bin/sh
HADOOP_CONF=/etc/hadoop/conf
while [ $# -gt 0 ] ; do
   nodeArg=$1
   exec< ${HADOOP_CONF}/topology.data
   result=""
   while read line ; do
     ar=( $line )
     if [ "${ar[0]}" = "$nodeArg" ] ; then
       result="${ar[1]}"
     fi
   done
   shift
   if [ -z "$result" ] ; then
     echo -n "/default/rack "
   else
     echo -n "$result "
   fi
done


The topology data file looks like so:

192.168.8.50    /dc1/rack1
192.168.8.70    /dc1/rack2
192.168.8.90    /dc1/rack3
...

HTH,

DR

Mime
View raw message