hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-1226) makeQualified should return an instance of a DfsPath when passed a DfsPath
Date Tue, 22 May 2007 18:39:16 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12497951

Doug Cutting commented on HADOOP-1226:

> Please comment on whether keeping the task output path as a fullyQualified path [...]

Sigh.  That code shouldn't be there in the first place.  I was asleep when HADOOP-1127 was
committed.  We should try to keep the mapred kernel from touching input or output files, leaving
that to InputFormats and OutputFormats.  It's a church/state thing.  (Also, taskOutputPath
stuff is reduce specific, so if anywhere, it belongs in ReduceTask, no?)

File renaming for speculative reduce belongs in OutputFormatBase (which should really be called
FileOutputFormat).  We'll need to add another method to the OutputFormat interface, something
like complete(JobConf, String tempName, String finalName), that's called for each task to
rename its temporary output when speculative reduces are enabled, where tempName is what was
passed to getRecordWriter().  I'll file an issue for this.

Back to your question: I don't know why relative paths are used there.  Perhaps Arun does,
since he wrote that code as a part of HADOOP-1127.

> makeQualified should return an instance of a DfsPath when passed  a DfsPath 
> ----------------------------------------------------------------------------
>                 Key: HADOOP-1226
>                 URL: https://issues.apache.org/jira/browse/HADOOP-1226
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>            Reporter: Koji Noguchi
>         Assigned To: dhruba borthakur
>             Fix For: 0.14.0
>         Attachments: makeQualified2.patch
> Currently, it returns a instance of a Path.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message