hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ted Malaska (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5132) Enhance FairScheduler queue selection with Unix-SubGroups
Date Fri, 05 Apr 2013 18:15:16 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5132?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13623886#comment-13623886

Ted Malaska commented on MAPREDUCE-5132:

Just had a great talk with Karthik Kambatla and we have a starting design and approach.

Design:  We are going to make Sub-Groups work like user-name does today.  Because user-name
doesn't pull from the job config because of kerberos and security issues. 

1. I am going to assign the JIRI to my self.
2. I am going to implement a truck (MR2-YERN) solution first.  Then back port that solution
to branch-1 (MR1)
3. I will try to get the first patch done by end of week next week.


> Enhance FairScheduler queue selection with Unix-SubGroups
> ---------------------------------------------------------
>                 Key: MAPREDUCE-5132
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5132
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Ted Malaska
>            Priority: Minor
>         Attachments: FairSchedulerPlusCode.zip
> I have clients that use the primary group for other reasons and would like to select
the fairScheduler queue by subgroup.
> Currently the fairscheduler can only link a job config property to a fairscheduler queue
and subgroups are not in the job config.  
> Now thankfully we can get this data from UserGroupInformation.createProxyUser which is
cool because this is how TT currently see if it can copy data when kerberos is enabled.  Once
we have the sub-groups we just connect the first sub-group that matches a queue.
> I'll put up some code that I used to solve the problem in the short term.  
> Note that there is an issue with this code.  It doesn't populate the TT Web UI drop down
menu correctly.  I think I have a solution for that, but before I implement that I was wondering
if the community has any feedback first.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message