hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Collins <...@cloudera.com>
Subject Re: FileSystem contract of listStatus
Date Wed, 02 Nov 2011 23:25:01 GMT
Hey Noah,

HDFS returns items in lexographic order by byte (see
INode#compareBytes) but I don't think ordering was intended to be an
explicit part of the contract. Ie the test probably just needs to be
modified to ignore the order.

RawLocalFileSystem uses Java's File#list which has "no guarantee that
the name strings in the resulting array will appear in any specific
order; they  are not, in particular, guaranteed to appear in
alphabetical order.", however the FSContractBaseTest isn't run against
local file systems which is why it probably never came up.

Thanks,
Eli

On Wed, Nov 2, 2011 at 3:57 PM, Noah Watkins <jayhawk@soe.ucsc.edu> wrote:
> I have a question about the FileSystem contract in 0.20.
>
> In FileSystemContractBaseBaseTest:testFileStatus() there
> are several files created, and afterwards the test confirms
> that they are present. Here is the relevant code:
>
>    FileStatus[] paths = fs.listStatus(path("/test"));
>
>    paths = fs.listStatus(path("/test/hadoop"));
>    assertEquals(3, paths.length);
>    assertEquals(path("/test/hadoop/a"), paths[0].getPath());
>    assertEquals(path("/test/hadoop/b"), paths[1].getPath());
>    assertEquals(path("/test/hadoop/c"), paths[2].getPath());
>
> This test will fail if the results are not in the specific
> order. Is this ordering (alphanumeric?) part of the contract?
> Can FileSystem return results from listStatus() in any order?
>
> Thanks,
> Noah
>

Mime
View raw message