hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13926) S3Guard: Improve listLocatedStatus and listFiles
Date Tue, 07 Mar 2017 16:21:38 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15899684#comment-15899684
] 

Steve Loughran commented on HADOOP-13926:
-----------------------------------------

Looking at this some more, I'm happy with the limited-scalability list-to-array code going
into the initial preview release, HADOOP-13998. 

I would like it if we can keep the code in S3AFileSystem under control, especially given that
multiple operations will want to get at roughly the same data. I'd prefer for it to go into
S3Guard. 

There's also that little "TODO" about auth vs non-auth. That needs to be resolved because
it's quite a serious issue if there is a disconnect. Presumably: non-auth goes to the filesystem;
only auth can list on the DDB.

> S3Guard: Improve listLocatedStatus and listFiles
> ------------------------------------------------
>
>                 Key: HADOOP-13926
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13926
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Rajesh Balamohan
>            Assignee: Steve Loughran
>            Priority: Minor
>         Attachments: HADOOP-13926.wip.proto.branch-13345.1.patch
>
>
> Need to check if {{listLocatedStatus}} can make use of metastore's listChildren feature.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message