hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Boris Shkolnik (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1430) JobTracker should be able to renew delegation tokens for the jobs
Date Tue, 16 Feb 2010 18:13:28 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834358#action_12834358
] 

Boris Shkolnik commented on MAPREDUCE-1430:
-------------------------------------------


bq.1) The class DelegationTokenRenewal could be named better
It seems logical to me. If you have better suggestions please share.
bq. 2) I am having trouble understanding what you have done in DelegationTokenRenewal to do
with resetting of timers. When the JobTracker first renews the token, it can check when the
token is supposed to be renewed again, and create a corresponding TimerTask. Then cancel the
TimerTask when the job is done. The Timer object needs to be created only once, right?
This way you will create multiple timer objects. In the current design only one timer object
is created  - for the next expiring token.
bq. 3) You should have the cancellation of tokens based on a boolean configuration like "mapreduce.job.cancel.delegation.tokens".
 this is a new requirement. I will add it. I assume is the value is 'false' - we don't cancel
any tokens for the job? (not registering them up for renewal).
bq. 4) The KIND defined in DelegationTokenRenewal should come from the DelegationTokenIdentifier
in HDFS
it is package private. One would need to modify HDFS for it.
bq. 5) The place where the delegation tokens are registered for renewal is not correct. Note
that the JobInProgress constructor uses a token to copy the jobconf file from the hdfs and
by then the token for accessing the jobconf should have got renewed. The renewal should happen
before the JobInProgress is constructed in submitJob.
I don't think it is critical. Tokens don't really need to be renewed immediately. So the token
is perfectly valid at this point.
bq. 6) Also, now that we have MapReduce delegation tokens, it will be nice to have the class
DelegationTokenRenewal not assume only hdfs delegation tokens.
Well, we talked about it - and I understood that we DON'T want to renew MapReduce tokens in
JobTracker.
bq.7) The Timer should be cancelled when the JobTracker is stopped.
Ok. Will do.

> JobTracker should be able to renew delegation tokens for the jobs
> -----------------------------------------------------------------
>
>                 Key: MAPREDUCE-1430
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1430
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: jobtracker
>    Affects Versions: 0.22.0
>            Reporter: Devaraj Das
>            Assignee: Boris Shkolnik
>             Fix For: 0.22.0
>
>         Attachments: MAPREDUCE-1430-5.patch
>
>
> JobTracker should automatically renew delegation tokens for the jobs it is currently
running.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message