hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-299) maps from second jobs will not run until the first job finishes completely
Date Tue, 13 Jun 2006 21:24:29 GMT
maps from second jobs will not run until the first job finishes completely
--------------------------------------------------------------------------

         Key: HADOOP-299
         URL: http://issues.apache.org/jira/browse/HADOOP-299
     Project: Hadoop
        Type: Bug

  Components: mapred  
    Versions: 0.3.2    
    Reporter: Owen O'Malley
 Assigned to: Owen O'Malley 
     Fix For: 0.4.0


Because of the logic in the JobTracker's pollForNewTask, second jobs will rarely start running
maps until the first job finishes completely. The JobTracker leaves room to re-run failed
maps from the first job and it reserves the total number of maps for the first job. Thus,
if you have more maps in the first job than your cluster capacity, none of the second job
maps will ever run.

I propose setting the reserve to 1% of the first job's maps.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message