hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Naganarasimha Garla <naganarasimha...@gmail.com>
Subject Re: Communication in Hadoop YARN?
Date Wed, 16 Sep 2015 17:13:00 GMT
** transfer data between map and reduces : *Reducer doesn't fetch the data
directly from the map there is basically shuffle copy phase where reducer
connects to the shuffle handler aux service running as part of NM through
Http to copy the Map output data.
* *node managers contact the resource manager : *This communication
basically happens through RPC using proto bufs.

+ Naga

On Wed, Sep 16, 2015 at 8:01 PM, xeonmailinglist <xeonmailinglist@gmail.com>

> Hi,
> Hadoop YARN uses HDFS to store and read data from the filesystem. But what
> communication technology uses to transfer data between map and reduces, and
> for the node managers contact the resource manager? All communication in
> point to point?
> Thanks,

View raw message