Allen Wittenauer wrote:
> On 3/3/10 5:01 PM, "Mag Gam" <magawake@gmail.com> wrote:
>
>> Thanks Alan! Your presentation is very nice!
>
> Thanks. :)
>
>> "If you don't provide a script for rack awareness, it treats every
>> node as if it was its own rack". I am using the default settings and
>> the report still says only 1 rack.
>
> Let's take a different approach to convince you. :)
>
> Think about the question: Is there a difference between all nodes in one
> rack vs. every node acting as a lone rack?
>
> The answer is no, there isn't any difference. In both cases, all copies of
> the blocks can go to pretty much any node. When a MR job runs, every node
> would either be considered 'off rack' or 'rack-local'.
>
> So there is no difference.
>
>
>> Do you mind sharing a script with us on how you determine a rack? and
>> a sample <configuration> </configuration> syntax?
>
> Michael has already posted his, so I'll skip this one. :)
>
Think Mag probably wanted a shell script.
Mag, give your machines IPv4 addresses that map to rack number. 10.1.1.*
for rack one, 10.1.2.* for rack 2, etc. Then just filter out the IP
address by the top bytes, returning "10.1.1" for everything in rack one,
"10.1.2" for rack 2; Hadoop will be happy
|