hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Rawson <ryano...@gmail.com>
Subject Re: Hadoop node's locality
Date Mon, 09 Aug 2010 22:15:48 GMT
Hi,

The map-reduce scheduling does take block placement into account and
attempts to schedule map tasks accordingly.  The system is flexible,
and HBase uses it to put maps colocated with the regionservers they
read from.

-ryan

On Mon, Aug 9, 2010 at 2:32 PM, Renato Marroquín Mogrovejo
<renatoj.marroquin@gmail.com> wrote:
> Hi, thanks for answering, but I did find the answer out. I knew Hadoop did
> it, but I needed a reference.
> I found it in "Data-intensive Text Processing with MapReduce" in the page
> 25.
> I quoted here:
>
> Data/code co-location: . . . "An important optimization here is to prefer
> nodes that are on the same rack in the datacenter as the node holding the
> relevant data block, since inter-rack bandwidth is significantly less than
> intra-rack bandwidth."
>
> Thanks again.
>
>
> Renato M.
>
>
> 2010/8/9 vaibhav negi <sssssssenator@gmail.com>
>
>> Hi ,
>>
>> I am also a nebie to hadoop. But i think u should read about "rack aware"
>> concept.
>> It may solve your query.
>>
>> Vaibhav Negi
>>
>>
>> On Sun, Aug 8, 2010 at 12:10 AM, Renato Marroquín Mogrovejo <
>> renatoj.marroquin@gmail.com> wrote:
>>
>> > Hi everyone,
>> > Does anybody know if Hadoop can take scheduling decisions using nodes
>> > positions
>> > within network topology ? Or if there is any work being done on this?
>> > Thanks in advanced.
>> >
>> >
>> > Renato M.
>> >
>>
>

Mime
View raw message