hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1354) Incremental enhancements to the JobTracker for better scalability
Date Mon, 10 May 2010 06:35:50 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1354?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12865672#action_12865672
] 

Amareshwari Sriramadasu commented on MAPREDUCE-1354:
----------------------------------------------------

Some comments on the patch:
* patch would need changes in TaskDataView.java for corresponding computeNumSlotsPerReduce
changes in 0.20 patch. But the same changes are present in MAPREDUCE-1533. So, this comment
can be ignored as well.
* Any reason for making JobTracker.getNewJobID() un-synchrozied? I see deprecated getNewJobId()
still synchronized.
* In JobTracker.submitJob(), the following code change 
{code}
+      if (ugi == null) {
+       ugi = UserGroupInformation.getCurrentUser();
+      }
{code}
is not needed. ugi would never be null. 

Other changes look fine to me.

> Incremental enhancements to the JobTracker for better scalability
> -----------------------------------------------------------------
>
>                 Key: MAPREDUCE-1354
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1354
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>            Reporter: Devaraj Das
>            Assignee: Arun C Murthy
>            Priority: Critical
>         Attachments: mapreduce-1354--2010-03-10.patch, MAPREDUCE-1354_yhadoop20.patch,
MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch,
MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch, MAPREDUCE-1354_yhadoop20.patch,
mr-1354-y20.patch
>
>
> It'd be nice to have the JobTracker object not be locked while accessing the HDFS for
reading the jobconf file and while writing the jobinfo file in the submitJob method. We should
see if we can avoid taking the lock altogether.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message