hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hairong Kuang (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-4717) Removal of default port# in NameNode.getUri() cause a map/reduce job failed to prompt temporay output
Date Mon, 24 Nov 2008 19:13:44 GMT
Removal of default port# in NameNode.getUri() cause a map/reduce job failed to prompt temporay
output
-----------------------------------------------------------------------------------------------------

                 Key: HADOOP-4717
                 URL: https://issues.apache.org/jira/browse/HADOOP-4717
             Project: Hadoop Core
          Issue Type: Bug
          Components: dfs
    Affects Versions: 0.18.0
            Reporter: Hairong Kuang
             Fix For: 0.18.3


Problem reported here is that when the default port number (8020) is specified in the output,
job succeeds but no output is created. The cause of the problem is that "listStatus" call
drops the port number because NameNode.getUri removes the default port#.

Assuming that a map/reduce output directory is set to be "hdfs://localhost:8020/out", A call
"listStatus" on any of its sub directory, for example, "hdfs://localhost:8020/out/tempXX",
returns results like below: 

hdfs://localhost/out/tempXX/part-00005

Because of this, Task.java
    574   private Path getFinalPath(Path jobOutputDir, Path taskOutput) {
    575     URI relativePath = taskOutputPath.toUri().relativize(taskOutput.toUri());

does not get the correct relativePath because TaskOutputPath contain ports, but taskOutput
doesn't.

It seems to me that the problem could be fixed if we make Path.makeQualified() to return the
same path not matter the input path contains the default port or not. 


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message