hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Schubert Zhang <zson...@gmail.com>
Subject Re: Optimized Hadoop
Date Mon, 20 Feb 2012 18:17:31 GMT
We just update the slides of this improvements:
http://www.slideshare.net/hanborq/hanborq-optimizations-on-hadoop-mapreduce-20120216a

Updates:
(1) modified some describes to make things more clear and accuracy.
(2) add some benchmarks to make sense.

On Sat, Feb 18, 2012 at 11:12 PM, Anty <anty.rao@gmail.com> wrote:

>
>
> On Fri, Feb 17, 2012 at 3:27 AM, Todd Lipcon <todd@cloudera.com> wrote:
>
>> Hey Schubert,
>>
>> Looking at the code on github, it looks like your rewritten shuffle is
>> in fact just a backport of the shuffle from MR2. I didn't look closely
>>
>
> additionally, the rewritten shuffle in MR2 has some bugs, which harm the
> overall performance, for which I have already file a jira to report this,
> with a patch available.
> MAPREDUCE-3685 <https://issues.apache.org/jira/browse/MAPREDUCE-3685>
>
>
>
>> - are there any distinguishing factors?
>> Also, the OOB heartbeat and adaptive heartbeat code seems to be the
>> same as what's in 1.0?
>>
>> -Todd
>>
>> On Thu, Feb 16, 2012 at 9:44 AM, Schubert Zhang <zsongbo@gmail.com>
>> wrote:
>> > Here is the presentation to describe our job,
>> >
>> http://www.slideshare.net/hanborq/hanborq-optimizations-on-hadoop-mapreduce-20120216a
>> > Wellcome to give your advises.
>> > It's just a little step, and we are continue to do more improvements,
>> thanks
>> > for your help.
>> >
>> >
>> >
>> >
>> > On Thu, Feb 16, 2012 at 11:01 PM, Anty <anty.rao@gmail.com> wrote:
>> >>
>> >> Hi: Guys
>> >>        We just deliver a optimized hadoop , if you are interested, Pls
>> >> refer to https://github.com/hanborq/hadoop
>> >>
>> >> --
>> >> Best Regards
>> >> Anty Rao
>> >
>> >
>>
>>
>>
>> --
>> Todd Lipcon
>> Software Engineer, Cloudera
>>
>
>
>
> --
> Best Regards
> Anty Rao
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message