hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Habermaas (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-5610) When connecting to HDFS using an IP Task MapRed gets confused when checking the output path.
Date Wed, 01 Apr 2009 19:06:13 GMT
When connecting to HDFS using an IP  Task MapRed gets confused when checking the output path.


                 Key: HADOOP-5610
                 URL: https://issues.apache.org/jira/browse/HADOOP-5610
             Project: Hadoop Core
          Issue Type: Bug
          Components: mapred
    Affects Versions: 0.19.1, 0.19.0, 0.18.3
         Environment: AIX 6.1 and Solaris
            Reporter: Bill Habermaas
            Priority: Minor

The exception appearing below happens if you are using an IP address to reference a HDFS file
system path.  Although the URI authority for either path (whether referenced by hostname:port
or IP-address:port) refer to the same filesystem, Task throws an exception because the stringified
comparisons of the authority in the URIs are not equal.  The FileSystem holds the network
location of the namenode as a host name regardless as to whether an IP address or hostname
is used when the file system is connected (refer to HADOOP-5191 for an earlier fix specifically
for HDFS only). 

2009-03-27 04:15:45,045 WARN [Thread-145] org.apache.hadoop.mapred.LocalJobRunner: job_local_0002
java.io.IOException: Can not get the relative path: base = hdfs://
child = hdfs://p520aix61.mydomain.com:9000/mydata/2009/03/27/0bab100a-1bf1-499a-935d-bc4b4e94f44c/_temporary/_attempt_local_0002_r_000000_0/part-00000
at org.apache.hadoop.mapred.Task.getFinalPath(Task.java:586)
at org.apache.hadoop.mapred.Task.moveTaskOutputs(Task.java:599)
at org.apache.hadoop.mapred.Task.moveTaskOutputs(Task.java:617)
at org.apache.hadoop.mapred.Task.saveTaskOutput(Task.java:561)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:202)

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message