hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-52) mapred input and output dirs must be absolute
Date Wed, 22 Mar 2006 22:18:10 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-52?page=comments#action_12371471 ] 

Doug Cutting commented on HADOOP-52:

The 'synchronize (fs) ...' is worthless unless everyone does it, which they don't, so I wouldn't

I think you're right that my concern about LocalJobRunner is misplaced.  It is not a public
class, and a new instance is created for each job, so there's currently no way to have multiple
jobs sharing a LocalJobRunner.

Longer term, if this becomes an issue, I think making FileSystem cloneable is preferable to

> mapred input and output dirs must be absolute
> ---------------------------------------------
>          Key: HADOOP-52
>          URL: http://issues.apache.org/jira/browse/HADOOP-52
>      Project: Hadoop
>         Type: Bug
>   Components: mapred
>     Versions: 0.1
>     Reporter: Doug Cutting
>     Assignee: Owen O'Malley
>      Fix For: 0.1
>  Attachments: cwd.patch
> DFS converts relative pathnames to be under /user/$USER.  But MapReduce jobs may be submitted
by a different user than is running the jobtracker and tasktracker.  Thus relative paths must
be resolved before a job is submitted, so that only absolute paths are seen on the job tracker
and tasktracker.  I think the simplest way to fix this is to make JobConf.setInputDir(), setOutputDir(),
etc. resolve relative pathnames. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators:
For more information on JIRA, see:

View raw message