hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3698) Implement access control for submitting jobs to queues in the JobTracker
Date Mon, 01 Sep 2008 11:46:44 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12627443#action_12627443

Hemanth Yamijala commented on HADOOP-3698:

I had an offline discussion with Owen on his review comments. Here's the summary, and explanation
of the changes in the patch with respect to these.

bq. The JobClient should use UserGroupInformation.login(conf) rather than UnixUserGroupInformation.login(conf).

The JobClient code was already using {{UnixUserGroupInformation.login(conf, saveToConf)}}.
There was no API equivalent to this one in UserGroupInformation. Discussing options, we decided
it's best to have a new JIRA to address this issue in UserGroupInformation. I will open the
JIRA and link it off here. Until then, because this issue is holding up other JIRAs, we decided
to retain the current code as is.

bq. I think we should have an exception class to catch for login failures. So we either need
to add LoginException to submitJob in JobClient or we should create a new IOException that
is for login failures and put it into org.apache.hadoop.security. I'd lean toward changing
the signature of run/submitJob.

Because run/submitJob throwing a new Exception would cause backwards incompatibility, we decided
to go with the other option, that is create a sub-class of IOException in org.apache.hadoop.security.
Looking at it, I've seen there's already an {{AccessControlException}} in org.apache.hadoop.fs.permission.
Since both classes are identical, I think we should leave one (possibly the one being defined
here), and deprecate and ultimately replace the other. This could be part of the cleanup JIRA
being opened above. Thoughts ?
Also, please note that UserGroupInformation still throws only LoginException. This seems fair
because the calls that generate this are no remote calls, and hence don't need to throw any
IOException or any sub-class of it.

bq. I'd suggest handling mapred.acls.enabled inside of the QueueManager instead of the JobTracker.
The same for giving the user access to their own jobs. That should be done in QueueManager,
so that all of the rules & exceptions for allowing actions are contained inside the one

Done this. I've defined a new API in QueueManager to handle this.
boolean hasAccess(String queueName, JobInProgress job, UserGroupInformation user, QueueOperation
action) { ... }

bq. The JobTracker should use the standard method for getting the queue of the job instead
of using the attribute name "queue.name".

This was being done in HADOOP-3445. Still, I've defined these changes in the JobConf and JobProfile
classes to get the queue name of a job. It defaults to "default" - which is the default queue
defined in the queue manager. This change might need this JIRA to be committed before HADOOP-3445.

bq. Why does the TaskScheduler need the QueueManager? That seems really unexpected.

This was done as discussed in HADOOP-3930 [here|https://issues.apache.org/jira/browse/HADOOP-3930?focusedCommentId=12626042#action_12626042].
However, Owen suggested that this be defined as part of the TaskTrackerManager interface,
and define a {{getQueueManager}} in the JobTracker. Implemented that change.

> Implement access control for submitting jobs to queues in the JobTracker
> ------------------------------------------------------------------------
>                 Key: HADOOP-3698
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3698
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: mapred
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
>             Fix For: 0.19.0
>         Attachments: HADOOP-3698.patch, HADOOP-3698.patch, HADOOP-3698.patch
> HADOOP-3445 implements multiple queues in the JobTracker as part of the new resource
manager for Hadoop (HADOOP-3421). There needs to be a mechanism to control who can submit
jobs to a specified queue. This JIRA is for tracking the requirements, approach and implementation
for the same.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message