hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1430) JobTracker should be able to renew delegation tokens for the jobs
Date Sun, 14 Feb 2010 18:27:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12833604#action_12833604

Devaraj Das commented on MAPREDUCE-1430:

1) The class DelegationTokenRenewal could be named better
2) I am having trouble understanding what you have done in DelegationTokenRenewal to do with
resetting of timers. When the JobTracker first renews the token, it can check when the token
is supposed to be renewed again, and create a corresponding TimerTask. Then cancel the TimerTask
when the job is done. The Timer object needs to be created only once, right?
3) You should have the cancellation of tokens based on a boolean configuration like "mapreduce.job.cancel.delegation.tokens".

4) The KIND defined in DelegationTokenRenewal should come from the DelegationTokenIdentifier
5) The place where the delegation tokens are registered for renewal is not correct. Note that
the JobInProgress constructor uses a token to copy the jobconf file from the hdfs and by then
the token for accessing the jobconf should have got renewed. The renewal should happen before
the JobInProgress is constructed in submitJob.
6) Also, now that we have MapReduce delegation tokens, it will be nice to have the class DelegationTokenRenewal
not assume only hdfs delegation tokens.
7) The Timer should be cancelled when the JobTracker is stopped.

> JobTracker should be able to renew delegation tokens for the jobs
> -----------------------------------------------------------------
>                 Key: MAPREDUCE-1430
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1430
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: jobtracker
>    Affects Versions: 0.22.0
>            Reporter: Devaraj Das
>            Assignee: Boris Shkolnik
>             Fix For: 0.22.0
>         Attachments: MAPREDUCE-1430-5.patch
> JobTracker should automatically renew delegation tokens for the jobs it is currently

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message