hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Uma Maheswara Rao G 72686 <mahesw...@huawei.com>
Subject Re: FileSystem contract of listStatus
Date Sun, 06 Nov 2011 02:51:43 GMT
----- Original Message -----
From: "M. C. Srivas" <mcsrivas@gmail.com>
Date: Sunday, November 6, 2011 3:13 am
Subject: Re: FileSystem contract of listStatus
To: common-dev@hadoop.apache.org

> On Thu, Nov 3, 2011 at 4:27 AM, Uma Maheswara Rao G 72686 <
> maheswara@huawei.com> wrote:
> 
> > Yes, i remember this issue filed by Harsh recently.
> >  GlobStatus will sort the results and return. May be we can fix for
> > listStatus in the same way.
> >
> 
> Not a good idea to sort needlessly.  That's why we have 
> globStatus() and
> listStatus() ... those who want a sorted list can use globStatus().

globStatus is for pattern matching and listStatus is for listing all the files in given directory.

> 
> 
> 
> >
> > Regards,
> > Uma
> > ----- Original Message -----
> > From: Harsh J <harsh@cloudera.com>
> > Date: Thursday, November 3, 2011 7:52 am
> > Subject: Re: FileSystem contract of listStatus
> > To: common-dev@hadoop.apache.org
> >
> > > Perhaps fixes against this can be covered as part of
> > > https://issues.apache.org/jira/browse/HADOOP-7659
> > >
> > > On 03-Nov-2011, at 5:44 AM, Ted Dunning wrote:
> > >
> > > > I think that the API docs actually say globStatus is ordered and
> > > leave the
> > > > ordering semantics for listStatus undefined.
> > > >
> > > >
> > >
> > 
> http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/fs/FileSystem.html#globStatus(org.apache.hadoop.fs.Path)>
>
> > > >
> > >
> > 
> http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/fs/RawLocalFileSystem.html#listStatus(org.apache.hadoop.fs.Path)>
>
> > > > On Wed, Nov 2, 2011 at 4:57 PM, Noah Watkins
> > > <jayhawk@soe.ucsc.edu> wrote:
> > > >
> > > >> ----- Original Message -----
> > > >>> From: "Eli Collins" <eli@cloudera.com>
> > > >>>
> > > >>> RawLocalFileSystem uses Java's File#list which has "no
> > > guarantee that
> > > >>> the name strings in the resulting array will appear in any
> > > specific>>> order; they are not, in particular, guaranteed to
> > > appear in
> > > >>> alphabetical order.", however the FSContractBaseTest isn't run
> > > against>>> local file systems which is why it probably never 
> came up.
> > > >>
> > > >> Thanks Eli. We are cleaning up the unit tests for Ceph and the
> > > unit tests
> > > >> use an emulation layer built on top of the local FS. We ran
> > > into this
> > > >> ordering
> > > >> issue. Getting a fix to this would be nice.
> > > >>
> > > >> Thanks!
> > > >> -Noah
> > > >>
> > >
> > >
> >
> 
Regards,
Uma

Mime
View raw message