hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3975) Default value not set for Configuration parameter mapreduce.job.local.dir
Date Thu, 08 Mar 2012 19:05:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13225401#comment-13225401
] 

Robert Joseph Evans commented on MAPREDUCE-3975:
------------------------------------------------

Vinod, can we open a new JIRA for this?

The patch has already been merged in, and it is a step closer to what we want, although obviously
not all the way there.  If the directories are not the same the worst thing that happens is
that some resources are wasted. There are no guarantees that the tasks will run on the same
node, so any mapreduce jobs that use this need to be able to handle a task running by itself.
                
> Default value not set for Configuration parameter mapreduce.job.local.dir
> -------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-3975
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3975
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.1, 0.23.2
>            Reporter: Eric Payne
>            Assignee: Eric Payne
>            Priority: Blocker
>             Fix For: 0.23.2
>
>         Attachments: MAPREDUCE-3975-1.txt
>
>
> mapreduce.job.local.dir (formerly job.local.dir in 0.20) is not set by default. This
is a regression from 0.20.205.
> In 0.20.205, JobLocalizer.createWorkDir() constructs the "$mapred.local.dir/taskTracker/$user/jobcache/$jobid/work"
path based on $user and $jobid, and then sets TaskTracker.JOB_LOCAL_DIR in the job's JobConf.
> So far, I haven't found where this is done in 0.23. It could be that this is what should
be done by LocalJobRunner.setupChildMapredLocalDirs(), but I am still investigating.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message