hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "V.V.Chaitanya Krishna (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-6439) Shuffle deadlocks on wrong number of maps
Date Wed, 21 Apr 2010 03:07:52 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-6439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12859175#action_12859175
] 

V.V.Chaitanya Krishna commented on HADOOP-6439:
-----------------------------------------------

Ran tests of mapreduce and hdfs with the core jar built with this patch.
All tests passed except for TestTrackerDistributedCacheManager in mapreduce and TestFiHFlush
in hdfs. 
These tests failed even without this patch being applied.

> Shuffle deadlocks on wrong number of maps
> -----------------------------------------
>
>                 Key: HADOOP-6439
>                 URL: https://issues.apache.org/jira/browse/HADOOP-6439
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: conf
>    Affects Versions: 0.21.0, 0.22.0
>            Reporter: Owen O'Malley
>            Assignee: V.V.Chaitanya Krishna
>            Priority: Blocker
>             Fix For: 0.21.0, 0.22.0
>
>         Attachments: HADOOP-6439-1.patch, HADOOP-6439-2.patch, HADOOP-6439-3.patch, HADOOP-6439-4.patch,
HADOOP-6439-5.patch, HADOOP-6439-6.patch, HADOOP-6439-6.patch, HADOOP-6439-7.patch, mr-1252.patch
>
>
> The new shuffle assumes that the number of maps is correct. The new JobSubmitter sets
the old value. Something misfires in the middle causing:
> 09/12/01 00:00:15 WARN conf.Configuration: mapred.job.split.file is deprecated. Instead,
use mapreduce.job.splitfile
> 09/12/01 00:00:15 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use
mapreduce.job.maps
> But my reduces got stuck at 2 maps / 12 when there were only 2 maps in the job.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message