hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1595) LinuxTaskController is too strict on the initial ownership of files/dir.
Date Fri, 12 Mar 2010 10:51:27 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12844452#action_12844452

Vinod K V commented on MAPREDUCE-1595:

Consider this failure case:
 - Two mapred-local-dirs "/a/b/c/ML1" and "/a/b/c/ML2".
 - TT first creates userdir Alice in both the dirs, i.e. "/a/b/c/ML1/taskTracker/Alice" and
 - TT then launched LinuxTaskController for 'securifying' the user-dirs aka MAPREDUCE-856.
 - LinuxTaskController then successfully changes ownership of "/a/b/c/ML1/taskTracker/Alice"
to "Alice:tt_group" but then fails on changing ownership of "/a/b/c/ML2/taskTracker/Alice"
due to a transitory disk problem.

The above result in the failure of current task because of a failed INITIALIZE_USER operation.
Not just that, every other task of Alice that ever comes on this TT will try INITIALIZE_USER
and fail because LinuxTaskController see "Alice:tt_group" on "/a/b/c/ML1/taskTracker/Alice"
and fails saying the file is not owned by TT.

This can otherwise only be rectified when TT restarts/reinits and moves and deletes the old

> LinuxTaskController is too strict on the initial ownership of files/dir.
> ------------------------------------------------------------------------
>                 Key: MAPREDUCE-1595
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1595
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: security, task-controller
>            Reporter: Vinod K V
>             Fix For: 0.22.0
> Linux task controller is too strict now w.r.t the initial ownership of the files/dir
that it tries to make as secure as possible. Currently, it expects, for e.g, the mapred-local/tasktracker/user-dir
to be both user-owned and group-owned by TT. This leads to unrecoverable failures in some
corner cases.
> It can instead allow the files/dirs to be owned either by TT *or* by the jobOwner.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message