hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13926) S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore
Date Mon, 03 Apr 2017 17:01:41 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15953818#comment-15953818
] 

Steve Loughran commented on HADOOP-13926:
-----------------------------------------

LGTM, only a couple of nits

h3. {{Listing}}


* L225. could we have an error message which is less likely to be mistaken for the user being
told off? "Null status list" would work.


h3. {{S3AFileSystem}}

Line 2503: can we have java 7 code; maybe a new Acceptor in the Listing class to substitute
for a (nice)) lambda expression?



I think we are all agreed that this is an interim feature for initial previews, as a production
one will need to do the DDB queries as we go along.

At some point then, all the params which are passed down to listing, or at least the array,
is going to change to something else, such
as an iterator. I'm wondering whether this can be adopted today  in {{ProvidedLocatedFileStatusIterator}}
just by taking an Iterator from the outset.
I was thinking we could also do filtering internally by way of Guava's {{Iterator}} helpers,
but that won't be the case will it: they aren't for
RemoteIterator.

Conclusion: probably over-complex right now, this isn't a public API. We can do the more elegant
solution once we know what it is we are trying to do.

> S3Guard: S3AFileSystem::listLocatedStatus() to employ MetadataStore
> -------------------------------------------------------------------
>
>                 Key: HADOOP-13926
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13926
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>            Reporter: Rajesh Balamohan
>            Assignee: Steve Loughran
>         Attachments: HADOOP-13926-HADOOP-13345.001.patch, HADOOP-13926-HADOOP-13345.002.patch,
HADOOP-13926-HADOOP-13345.003.patch, HADOOP-13926.wip.proto.branch-13345.1.patch
>
>
> Need to check if {{listLocatedStatus}} can make use of metastore's listChildren feature.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message