hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From brisk <mylinq...@gmail.com>
Subject Re: Optimized Hadoop
Date Tue, 28 Feb 2012 21:08:16 GMT
Hi Arun,

Just curious, could you please give a quick hint about why 0.23.1 performs
much better in shuffle phase? Any significant changes from prior version?

Thanks,
Ethan

On Thu, Feb 16, 2012 at 9:38 AM, Arun C Murthy <acm@hortonworks.com> wrote:

> Interesting, thanks for sharing.
>
> Are you planning on contributing any of these back to Apache Hadoop?
>
> Also, hadoop-0.23.1 significantly outperforms hadoop-1 (previously
> hadoop-0.20.xxx) on both framework (map/reduce task launch) and runtime
> (sort, shuffle etc.). Also makes it much easier to implement Distributed
> Worker Pool with the newer MRv2 (YARN) architecture. We would love to have
> you involved and to contribute back.
>
> Arun
>
> On Feb 16, 2012, at 7:01 AM, Anty wrote:
>
> > Hi: Guys
> >       We just deliver a optimized hadoop , if you are interested, Pls
> > refer to https://github.com/hanborq/hadoop
> >
> > --
> > Best Regards
> > Anty Rao
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message