hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: In memory shuffle phase
Date Wed, 12 Jun 2013 00:35:50 GMT
Take a look at Apache Tez (http://incubator.apache.org/projects/tez.html).

I'll warn you that it's a little rough, but works.

Arun

On Jun 10, 2013, at 5:46 PM, Grandl Robert wrote:

> Hi,
> 
> 
> It is possible to run shuffle phase in memory ? I mean, the mapper to write the intermediate
output in memory only and the reducer to read it from there. So no disk involved for intermediate
output. 
> 
> 
> If so, what are the configuration parameters I need to tune ?
> 
> Thanks,
> Robert

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message