hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joydeep Sen Sarma (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-2118) optimize getJobSetupAndCleanupTasks
Date Fri, 08 Oct 2010 17:02:31 GMT
optimize getJobSetupAndCleanupTasks 

                 Key: MAPREDUCE-2118
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2118
             Project: Hadoop Map/Reduce
          Issue Type: Improvement
            Reporter: Joydeep Sen Sarma

in every heartbeat, while holding the JobTracker global lock, all jobs are scanned for job
setup/cleanup, task setup/cleanup. on a large system with many trackers (and heartbeats) and
many jobs - this becomes the bottleneck for JT throughput.

One possible route may be to rework the code to not require the JT lock while asking the JIP
whether it has a setup/cleanup task. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message