hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Runping Qi (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3366) Shuffle/Merge improvements
Date Tue, 13 May 2008 16:49:55 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12596436#action_12596436
] 

Runping Qi commented on HADOOP-3366:
------------------------------------


It is clear that there is a large room for improvement on the merge code. I
Both map and reduce will benefits from the improvements. It makes sense to replace sequencefile.merge
with some simpler
and lighter and more efficient one.  Once 2095 is in, we may get a better picture where will
be the new
bottleneck then, and make due decisions on some areas.
Overall, the direction is clearly right.



> Shuffle/Merge improvements
> --------------------------
>
>                 Key: HADOOP-3366
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3366
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: mapred
>            Reporter: Arun C Murthy
>            Assignee: Arun C Murthy
>             Fix For: 0.18.0
>
>
> This is intended to be a meta-issue to track various improvements to shuffle/merge in
the reducer.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message