hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5047) keep.failed.task.files=true causes job failure on secure clusters
Date Mon, 11 Mar 2013 21:05:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13599295#comment-13599295
] 

Sandy Ryza commented on MAPREDUCE-5047:
---------------------------------------

To elaborate on this a little further, IsolationRunner throws a NPE whenever it starts due
to not reading in mapred.local.dir from the job conf before trying to use it.  IsolationRunner
has been broken for a long time and was removed in MR2 (MAPREDUCE-2606).

While this patch doesn't fix IsolationRunner, keep.failed.task.files still has utility without
it, and this fixes the bug that was causing jobs to fail every time it's turned on.
                
> keep.failed.task.files=true causes job failure on secure clusters
> -----------------------------------------------------------------
>
>                 Key: MAPREDUCE-5047
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5047
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: task, tasktracker
>    Affects Versions: 1.1.1
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>             Fix For: 1.3.0
>
>         Attachments: MAPREDUCE-5047.patch
>
>
> To support IsolationRunner, split info is written to local directories.  This occurs
inside MapTask#localizeConfiguration, which is called both tasktracker and by the child JVM.
 On a secure cluster, the tasktacker's attempt to write it fails, because the tasktracker
does not have permission to write to the user's directory. It is likely that the call to localizeConfiguration
in the tasktracker can be removed. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message