hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anty <anty....@gmail.com>
Subject Re: Optimized Hadoop
Date Sat, 18 Feb 2012 15:12:59 GMT
On Fri, Feb 17, 2012 at 3:27 AM, Todd Lipcon <todd@cloudera.com> wrote:

> Hey Schubert,
>
> Looking at the code on github, it looks like your rewritten shuffle is
> in fact just a backport of the shuffle from MR2. I didn't look closely
>

additionally, the rewritten shuffle in MR2 has some bugs, which harm the
overall performance, for which I have already file a jira to report this,
with a patch available.
MAPREDUCE-3685 <https://issues.apache.org/jira/browse/MAPREDUCE-3685>



> - are there any distinguishing factors?
> Also, the OOB heartbeat and adaptive heartbeat code seems to be the
> same as what's in 1.0?
>
> -Todd
>
> On Thu, Feb 16, 2012 at 9:44 AM, Schubert Zhang <zsongbo@gmail.com> wrote:
> > Here is the presentation to describe our job,
> >
> http://www.slideshare.net/hanborq/hanborq-optimizations-on-hadoop-mapreduce-20120216a
> > Wellcome to give your advises.
> > It's just a little step, and we are continue to do more improvements,
> thanks
> > for your help.
> >
> >
> >
> >
> > On Thu, Feb 16, 2012 at 11:01 PM, Anty <anty.rao@gmail.com> wrote:
> >>
> >> Hi: Guys
> >>        We just deliver a optimized hadoop , if you are interested, Pls
> >> refer to https://github.com/hanborq/hadoop
> >>
> >> --
> >> Best Regards
> >> Anty Rao
> >
> >
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>



-- 
Best Regards
Anty Rao

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message