hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Colin Patrick McCabe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10798) globStatus() does not return sorted list of files
Date Thu, 18 Jun 2015 00:37:02 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14590935#comment-14590935

Colin Patrick McCabe commented on HADOOP-10798:

bq. I'd like to go ahead and remove the sorting language from the API.


bq. There's no need to do the sort on shell-side since it's not being done already and hence
there should be no behavior change.

Disagree.  "ls" on UNIX has always returned entries in sorted order.  So does Hadoop's ls,
except in the special case where you are using a non-HDFS filesystem.  This is not a normal
case, so the discrepancy got overlooked.  But I think we should fix it now.

We don't need to be ultra-fast in the shell, so there seems to be no reason why we shouldn't
just add a sort.  At least that's my thinking right now.  What do you think?

> globStatus() does not return sorted list of files
> -------------------------------------------------
>                 Key: HADOOP-10798
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10798
>             Project: Hadoop Common
>          Issue Type: Bug
>    Affects Versions: 2.3.0
>            Reporter: Felix Borchers
>            Assignee: Colin Patrick McCabe
>            Priority: Minor
>              Labels: BB2015-05-TBR
>         Attachments: HADOOP-10798.001.patch
> (FileSystem) globStatus() does not return a sorted file list anymore.
> But the API says: " ... Results are sorted by their names."
> Seems to be lost, when the Globber Object was introduced. Can't find a sort in actual
> code to check this behavior:
> {code}
>         Configuration conf = new Configuration();
>         FileSystem fs = FileSystem.get(conf);
>         Path path = new Path("/tmp/" + System.currentTimeMillis());
>         fs.mkdirs(path);
>         fs.deleteOnExit(path);
>         fs.createNewFile(new Path(path, "2"));
>         fs.createNewFile(new Path(path, "3"));
>         fs.createNewFile(new Path(path, "1"));
>         FileStatus[] status = fs.globStatus(new Path(path, "*"));
>         Collection list = new ArrayList();
>         for (FileStatus f: status) {
>             list.add(f.getPath().toString());
>             //System.out.println(f.getPath().toString());
>         }
>         boolean sorted = Ordering.natural().isOrdered(list);
>         Assert.assertTrue(sorted);
> {code}

This message was sent by Atlassian JIRA

View raw message