hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Victor Hsieh <victorhs...@gmail.com>
Subject Re: org.apache.hadoop.fs.Path toString() bug
Date Fri, 22 Jan 2010 02:19:18 GMT
Hi Eli,

Sorry, I just realized this is my own mistake.  My original problem is
that -libjar doesn't work when running mapreduce jobs.  After debuging
for a while, I thought it's the Path URI problem.  But the problem is
actually my wrong CLASSPATH and -libjar.

Thanks,
Victor

On Fri, Jan 22, 2010 at 2:45 AM, Eli Collins <eli@cloudera.com> wrote:
> On Thu, Jan 21, 2010 at 12:27 AM, Victor Hsieh <victorhsieh@gmail.com> wrote:
>> Hi,
>>
>> I probably find a bug in org.apache.hadoop.fs.Path.  Could you please
>> verify it for me?
>>
>> I suppose the following expression returns "file:///foor/bar", but
>> what returned is actually "file:/foo/bar".
>>    new Path("file", "", "/foo/bar").toString()
>>
>> Another more realistic case
>>    FileSystem fs = new LocalFileSystem();
>>    Path path = new Path("/foo/bar").makeQualified(fs);
>>    path.toString()  // returns "file:/foo/bar"
>>
>> A possible solution is proposed (patch attached), but it breaks the
>> first test case of testSchema in TestPath.java, which is:
>>    assertEquals("foo:/bar", new Path("foo:/","/bar").toString());
>>
>> I'm not sure why this test case make sense.  Any suggestion?
>
> hadoop.fs.Path is not actually a file system Path, it's a thin wrapper
> around a URI (which has well-defined behavior).  So the above behavior
> is expected, eg "file:///foor/bar" normalizes to "file:/foo/bar",
> similarly to make /foo/bar fully qualified you have to specify a
> schema (which is why it takes a file system).
>
> Do you have an example where the current behavior is insufficient? Or
> are you just pointing out that hadoop.fs.Path has a confusing name?
> (in which case I agree).
>
> Thanks,
> Eli
>

Mime
View raw message