hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1354) Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
Date Tue, 16 Feb 2010 11:49:28 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834174#action_12834174
] 

Amareshwari Sriramadasu commented on MAPREDUCE-1354:
----------------------------------------------------

Other than the above optimizations I suggested in my previous comment, the code changes look
fine. 
I verified all the accesses to JobInProgress.conf and JobInProgress.getJobConf(); verified
that synchronized methods in JobInProgress are not called if the job is not initialized; 
JobInProgress constructor calls jobtracker.getJobTrackerMachine(), jobtracker.getInfoPort(),
jobtracker.getSystemDirectoryForJob(), jobtracker.getNumTaskCacheLevels(). All these are getters
and their values are initialized in JobTracker's constructor. So, calling them without JobTracker
lock is fine. JobInProgress constructor calls jobtracker.getInstrumentation().addPrepJob()
also. The method addPrepJob() is a synchronized method in implementation of Instrumentation.
Thus, calling JobInProgress constructor withour JobTracker lock is fine.

> Refactor JobTracker.submitJob to not lock the JobTracker during the HDFS accesses
> ---------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-1354
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>            Reporter: Devaraj Das
>            Assignee: Arun C Murthy
>            Priority: Critical
>         Attachments: MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch,
MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch,
MAPREDUCE-1354_yhadoop20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for
reading the jobconf file and while writing the jobinfo file in the submitJob method. We should
see if we can avoid taking the lock altogether.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message