hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun C Murthy (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-318) Refactor reduce shuffle code
Date Tue, 25 Aug 2009 07:15:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12747268#action_12747268
] 

Arun C Murthy commented on MAPREDUCE-318:
-----------------------------------------

bq. Document rationale of why all fetch threads will never stall i.e. will allow one thread
to go past always

The patch looks fine, could you comment on the bug you saw with a deadlocked shuffle you saw
once since I do see you've reverted back to 'original' patch wrt the fetcher threads and stalling?

> Refactor reduce shuffle code
> ----------------------------
>
>                 Key: MAPREDUCE-318
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-318
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Owen O'Malley
>            Assignee: Owen O'Malley
>         Attachments: HADOOP-5233_api.patch, HADOOP-5233_part0.patch, mapred-318-14Aug.patch,
mapred-318-20Aug.patch, mapred-318-24Aug.patch, mapred-318-common.patch
>
>
> The reduce shuffle code has become very complex and entangled. I think we should move
it out of ReduceTask and into a separate package (org.apache.hadoop.mapred.task.reduce). Details
to follow.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message