hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aishwarya Venkataraman <avenk...@cs.ucsd.edu>
Subject Re: Question regarding network data transfer
Date Sun, 29 May 2011 01:19:52 GMT
So how does reducer obtain the mapper's output ? Does it make a network call
and read data from mappers local storage or does the mapper send the data ?

Thanks,
Aishwarya

On Sat, May 28, 2011 at 6:08 PM, jagaran das <jagaran_das@yahoo.co.in>wrote:

> Mappers write to local persistance and reducers write to hdfs
>
> - Jagaran
>
>
>
> ________________________________
> From: Aishwarya Venkataraman <avenkata@cs.ucsd.edu>
> To: common-dev@hadoop.apache.org
> Sent: Sat, 28 May, 2011 4:11:13 PM
> Subject: Question regarding network data transfer
>
> Hello,
>
> Where exactly(Java class) does the network data transfer happen from a
> mapper to a reducer ? Mapper seems to be writing data to the HDFS. Does
> reducer read data from the mapper's HDFS over the network ? This is evading
> me. Thanks in advance for the help.
>
> Cheers,
> Aishwarya
>



-- 
Thanks,
Aishwarya Venkataraman
avenkata@cs.ucsd.edu
Graduate Student | Department of Computer Science
University of California, San Diego

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message