hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alejandro Abdelnur <t...@cloudera.com>
Subject Re: FileSystem contract of listStatus
Date Mon, 07 Nov 2011 19:15:57 GMT
IMO sorting is something the FS shell should do, not the FileSystem.

Thanks.

Alejandro

On Sat, Nov 5, 2011 at 7:51 PM, Uma Maheswara Rao G 72686
<maheswara@huawei.com> wrote:
> ----- Original Message -----
> From: "M. C. Srivas" <mcsrivas@gmail.com>
> Date: Sunday, November 6, 2011 3:13 am
> Subject: Re: FileSystem contract of listStatus
> To: common-dev@hadoop.apache.org
>
>> On Thu, Nov 3, 2011 at 4:27 AM, Uma Maheswara Rao G 72686 <
>> maheswara@huawei.com> wrote:
>>
>> > Yes, i remember this issue filed by Harsh recently.
>> >  GlobStatus will sort the results and return. May be we can fix for
>> > listStatus in the same way.
>> >
>>
>> Not a good idea to sort needlessly.  That's why we have
>> globStatus() and
>> listStatus() ... those who want a sorted list can use globStatus().
>
> globStatus is for pattern matching and listStatus is for listing all the files in given
directory.
>
>>
>>
>>
>> >
>> > Regards,
>> > Uma
>> > ----- Original Message -----
>> > From: Harsh J <harsh@cloudera.com>
>> > Date: Thursday, November 3, 2011 7:52 am
>> > Subject: Re: FileSystem contract of listStatus
>> > To: common-dev@hadoop.apache.org
>> >
>> > > Perhaps fixes against this can be covered as part of
>> > > https://issues.apache.org/jira/browse/HADOOP-7659
>> > >
>> > > On 03-Nov-2011, at 5:44 AM, Ted Dunning wrote:
>> > >
>> > > > I think that the API docs actually say globStatus is ordered and
>> > > leave the
>> > > > ordering semantics for listStatus undefined.
>> > > >
>> > > >
>> > >
>> >
>> http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/fs/FileSystem.html#globStatus(org.apache.hadoop.fs.Path)>
>
>> > > >
>> > >
>> >
>> http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/fs/RawLocalFileSystem.html#listStatus(org.apache.hadoop.fs.Path)>
>
>> > > > On Wed, Nov 2, 2011 at 4:57 PM, Noah Watkins
>> > > <jayhawk@soe.ucsc.edu> wrote:
>> > > >
>> > > >> ----- Original Message -----
>> > > >>> From: "Eli Collins" <eli@cloudera.com>
>> > > >>>
>> > > >>> RawLocalFileSystem uses Java's File#list which has "no
>> > > guarantee that
>> > > >>> the name strings in the resulting array will appear in any
>> > > specific>>> order; they are not, in particular, guaranteed to
>> > > appear in
>> > > >>> alphabetical order.", however the FSContractBaseTest isn't
run
>> > > against>>> local file systems which is why it probably never
>> came up.
>> > > >>
>> > > >> Thanks Eli. We are cleaning up the unit tests for Ceph and the
>> > > unit tests
>> > > >> use an emulation layer built on top of the local FS. We ran
>> > > into this
>> > > >> ordering
>> > > >> issue. Getting a fix to this would be nice.
>> > > >>
>> > > >> Thanks!
>> > > >> -Noah
>> > > >>
>> > >
>> > >
>> >
>>
> Regards,
> Uma
>

Mime
View raw message