hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1430) JobTracker should be able to renew delegation tokens for the jobs
Date Wed, 17 Feb 2010 08:40:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834691#action_12834691

Devaraj Das commented on MAPREDUCE-1430:

1) For (3), it should be that, cancellation is done for tokens owned by jobs that have mapreduce.job.cancel.delegation.tokens
set to true, when the job is done. The idea is to immediately cancel the tokens once the job
is done unless the user sets mapreduce.job.cancel.delegation.tokens to false (in which case
it probably means that the user launched jobs from tasks of the job in question, and these
new jobs would be using the same tokens for talking to the hdfs). For cancelling the token
you should invoke the appropriate RPC. Thinking about it, the config could be named to something
like "mapreduce.job.complete.cancel.delegation.tokens"

2) Please make the DelegationTokenRenewal class package private.

3) renewalTimer should be a daemon thread.

4) Methods like getDfsForToken could be declared as private.

5) The logic for setting the renewal to 60 minutes on an exception should be within the renewDelegationToken
method (newExpirationDate could be initialized to that).

6) You don't need synchronization on delegationTokens in the setTimerForTokenRenewal method.

7) The delegationTokens list could be a HashSet instead

Otherwise, looks good. I am still unhappy about the fact that you declare a string for "HDFS_DELEGATION_TOKEN"
within your class..

> JobTracker should be able to renew delegation tokens for the jobs
> -----------------------------------------------------------------
>                 Key: MAPREDUCE-1430
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1430
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: jobtracker
>    Affects Versions: 0.22.0
>            Reporter: Devaraj Das
>            Assignee: Boris Shkolnik
>             Fix For: 0.22.0
>         Attachments: MAPREDUCE-1430-5.patch, MAPREDUCE-1430-6.patch
> JobTracker should automatically renew delegation tokens for the jobs it is currently

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message