hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-673) the task execution environment should have a current working directory that is task specific
Date Mon, 20 Nov 2006 22:10:06 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-673?page=comments#action_12451450 ] 
            
Owen O'Malley commented on HADOOP-673:
--------------------------------------

Ok, I propose that we handle this as a streaming option. So:

1. For all map/reduce jobs the task has current working directory set to the task specific
directory.
2. For streaming if the property "mapred.create.symlink" is true, the contents of the job
directory (other than the task directories) are symlinked into the task directory. Note that
this property is already used for the file cache for a similar purpose and in streaming defaults
to true.

If there was a convenient way to make the job cache files read only in Java, that would make
me more comfortable about violating the sandbox isolation. But as it is, I guess it is upto
the application writer to make sure they don't change any of the shared files.

> the task execution environment should have a current working directory that is task specific
> --------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-673
>                 URL: http://issues.apache.org/jira/browse/HADOOP-673
>             Project: Hadoop
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.7.2
>            Reporter: Owen O'Malley
>         Assigned To: Mahadev konar
>             Fix For: 0.9.0
>
>
> The tasks should be run in a work directory that is specific to a single task. In particular,
I'd suggest using the <local>/jobcache/<jobid>/<taskid> as the current working
directory.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message