hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5154) staging directory deletion fails because delegation tokens have been cancelled
Date Wed, 17 Apr 2013 01:16:15 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13633637#comment-13633637

Sandy Ryza commented on MAPREDUCE-5154:

Uploading a patch that cancels the delegation tokens asynchronously as well.  This required
modifying CleanupQueue to accept delegation tokens to cancel in addition to files to delete.

Both TestJobRecovery and TestJobCleanup pass.
> staging directory deletion fails because delegation tokens have been cancelled
> ------------------------------------------------------------------------------
>                 Key: MAPREDUCE-5154
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5154
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>    Affects Versions: 1.2.0
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
> In a secure setup, the jobtracker needs the job's delegation tokens to delete the staging
directory.  MAPREDUCE-4850 made it so that job cleanup staging directory deletion occurs asynchronously,
so that it could order it with system directory deletion.  This introduced the issue that
a job's delegation tokens could be cancelled before the cleanup thread got around to deleting
it, causing the deletion to fail.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message