hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahadev Konar <maha...@hortonworks.com>
Subject Re: How is MRv2 fundamentally changed?
Date Mon, 16 Jan 2012 21:32:04 GMT
Hi Jie,
 You might want to read through:
http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/YARN.html
and http://developer.yahoo.com/blogs/hadoop/posts/2011/02/mapreduce-nextgen/

for more information on the architecture. Itll help you understand the
major differences between the two.

mahadev

On Mon, Jan 16, 2012 at 11:41 AM, Jie Li <jieli@cs.duke.edu> wrote:
> Hi all,
>
> As we know MRv2 (the MapReduce library in YARN) has changed significantly.
> We have a cost model built for the MapReduce in Hadoop and are going to
> migrate to MRv2. Can anyone give us a pointer to the fundamental
> differences between them? Also, below are some of my understandings and
> feel free to correct me.
>
> 1. JT has been replaced by a central RM and a per-application AM.
> 2. TT has been replaced by the NM and the task slots have been replaced by
> the containers. The containers can be allocated dynamically thus both the
> number and the memory size of the containers can vary on demand.
> 3. The shuffle service has become independent from the Map.
>
> Thanks,
> Jie

Mime
View raw message