hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zhijie Shen <zs...@hortonworks.com>
Subject Re: In memory shuffle phase
Date Thu, 13 Jun 2013 17:03:44 GMT
You may want to have a look at the following webpage:

http://hortonworks.com/blog/introducing-tez-faster-hadoop-processing/


On Wed, Jun 12, 2013 at 12:28 AM, 牛兆捷 <nzjemail@gmail.com> wrote:

> Is there any design document about tez?
>
> 2013/6/12 Arun C Murthy <acm@hortonworks.com>
>
> > Take a look at Apache Tez (http://incubator.apache.org/projects/tez.html
> ).
> >
> > I'll warn you that it's a little rough, but works.
> >
> > Arun
> >
> > On Jun 10, 2013, at 5:46 PM, Grandl Robert wrote:
> >
> > > Hi,
> > >
> > >
> > > It is possible to run shuffle phase in memory ? I mean, the mapper to
> > write the intermediate output in memory only and the reducer to read it
> > from there. So no disk involved for intermediate output.
> > >
> > >
> > > If so, what are the configuration parameters I need to tune ?
> > >
> > > Thanks,
> > > Robert
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/
> >
> >
> >
>
>
> --
> *Sincerely,*
> *Zhaojie*
> *
> *
>



-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message