hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Chen (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1829) JobInProgress.findSpeculativeTask should use min() to find the candidate instead of sort()
Date Tue, 08 Jun 2010 22:05:12 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12876831#action_12876831
] 

Scott Chen commented on MAPREDUCE-1829:
---------------------------------------

Run the failed contrib test on my box again. It succeed. And I don't think this is related
to the change.

{code}
Testsuite: org.apache.hadoop.mapred.TestSimulatorDeterministicReplay
Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 35.856 sec
------------- Standard Output ---------------
Job job_200904211745_0002 is submitted at 103010
Job job_200904211745_0002 completed at 141990 with status: SUCCEEDED runtime: 38980
Job job_200904211745_0003 is submitted at 984078
Job job_200904211745_0004 is submitted at 993516
Job job_200904211745_0003 completed at 1011051 with status: SUCCEEDED runtime: 26973
Job job_200904211745_0005 is submitted at 1033963
Done, total events processed: 595469
Job job_200904211745_0002 is submitted at 103010
Job job_200904211745_0002 completed at 141990 with status: SUCCEEDED runtime: 38980
Job job_200904211745_0003 is submitted at 984078
Job job_200904211745_0004 is submitted at 993516
Job job_200904211745_0003 completed at 1011051 with status: SUCCEEDED runtime: 26973
Job job_200904211745_0005 is submitted at 1033963
Done, total events processed: 595469
------------- ---------------- ---------------
{code}

> JobInProgress.findSpeculativeTask should use min() to find the candidate instead of sort()
> ------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1829
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1829
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: jobtracker
>    Affects Versions: 0.22.0
>            Reporter: Scott Chen
>            Assignee: Scott Chen
>             Fix For: 0.22.0
>
>         Attachments: MAPREDUCE-1829.txt
>
>
> findSpeculativeTask needs only one candidate to speculate so it does not need to sort
the whole list. It may looks OK but someone can still submit big jobs with small slow task
thresholds. In this case, this sorting becomes expensive.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message