hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patai Sangbutsarakum <silvianhad...@gmail.com>
Subject Re: Rack-awareness in Hadoop-2.0.3-alpha
Date Wed, 22 May 2013 17:55:41 GMT
I believe that his topology.sh and .data files are already correct.
bash topology.sh mustaqeem-1 mustaqeem-4
/default/rack /rack2 /rack3

output looks exactly the same as mine.

1. did you restart namenode after you modified the configuration? in 0.20,
restart namenode is required.
2. Try to revert the scheduler to the default i believe it is FIFO, and
play with Rack-awareness again.
3. Check if you need to chown topology.sh to the user that run namenode
4. Check if you need to chmod u+x topology.sh
5. Check if topology.data can be read by the user.

hope this help

On Wed, May 22, 2013 at 9:49 AM, Chris Nauroth <cnauroth@hortonworks.com>wrote:

> common-dev and hdfs-dev removed/bcc'd
> Hi Mohammad,
> Rack awareness is supported in 2.0.3-alpha.  The only potential problem I
> see in your configuration is that topology.sh contains a definition for
> HADOOP_CONF that points back at your hadoop-0.22.0/conf directory.  If that
> directory doesn't contain the right topology.data file, then resolution to
> a rack might not work as you expected.
> If that doesn't fix the problem, then I have another suggestion: change
> topology.sh to add some debugging statements that echo to a temp file the
> arguments that the script has received, and the results it decides to
> print.  This would show you exactly how Hadoop called your script and
> exactly what rack your script replied back to Hadoop.  It might give you an
> idea for what to investigate next.
> Hope this helps,
> Chris Nauroth
> Hortonworks
> http://hortonworks.com/
> On Wed, May 22, 2013 at 6:29 AM, Mohammad Mustaqeem <
> 3m.mustaqeem@gmail.com> wrote:
>> Is Hadoop-2.0.3-alpha does not support Rack-awarness?
>> I am trying to make Hadoop cluster Rack-Aware for a week but I
>> haven't succeed.
>> What I am doing.
>> I am adding following property in etc/hadoop/core-site.xml :
>> <property>
>> <name>net.topology.script.file.name</name>
>> <value>/home/hadoop/hadoop-2.0.3-alpha/etc/hadoop/topology.sh</value>
>> </property>
>> The topology.sh and topology.data is attached with this mail.
>> I want to tell you that once I used this property(
>> topology.script.file.name) and files to make Hadoop-0.22.0 rack-aware
>> but I am unable to do the same for Hadoop-2.0.3-alpha.
>> What I am doing wrong?
>> Please, somebody help me.
>> --
>> *With regards ---*
>> *Mohammad Mustaqeem*,
>> M.Tech (CSE)
>> MNNIT Allahabad

View raw message