hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-52) mapred input and output dirs must be absolute
Date Wed, 22 Mar 2006 22:32:01 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-52?page=comments#action_12371476 ] 

Owen O'Malley commented on HADOOP-52:
-------------------------------------

The synchronized (fs) will help the case where the application submits two jobs to the LocalJobRunner
with different working directories. Because the LocalJobRunner obeys the synchronization with
respect to the file system object, it should work. (I don't have any parallel map/reduce job
applications to test it with and testing for missing synchronization is almost impossible
anyways.)

> mapred input and output dirs must be absolute
> ---------------------------------------------
>
>          Key: HADOOP-52
>          URL: http://issues.apache.org/jira/browse/HADOOP-52
>      Project: Hadoop
>         Type: Bug
>   Components: mapred
>     Versions: 0.1
>     Reporter: Doug Cutting
>     Assignee: Owen O'Malley
>      Fix For: 0.1
>  Attachments: cwd.patch, cwd2.patch
>
> DFS converts relative pathnames to be under /user/$USER.  But MapReduce jobs may be submitted
by a different user than is running the jobtracker and tasktracker.  Thus relative paths must
be resolved before a job is submitted, so that only absolute paths are seen on the job tracker
and tasktracker.  I think the simplest way to fix this is to make JobConf.setInputDir(), setOutputDir(),
etc. resolve relative pathnames. 

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
   http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see:
   http://www.atlassian.com/software/jira


Mime
View raw message