hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anu Engineer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11569) Ozone: Implement listKey function for KeyManager
Date Wed, 05 Apr 2017 19:57:41 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15957544#comment-15957544
] 

Anu Engineer commented on HDFS-11569:
-------------------------------------

[~cheersyang] Thanks for the patch. Sorry for the delay in looking at this.

* Just to make sure that we are all on the same page.
ListKey function has to be invoked via 
{{DistributedStorageHandler#ListKeys}} -- We don't have do it in this patch. But wanted to
make
sure that we are on the same page.
  @Override
  public ListKeys listKeys(ListArgs args) throws IOException, OzoneException {
    throw new UnsupportedOperationException("listKeys not implemented");
  }
This function is invoked each time we make an HTTP request to the Ozone Web Handler.
Unfortunately, we cannot hold on to the result state on the HTTP front end. We need the client
that is talking to us holding the state, so if you have ListKeyResult and it has has more
pages to return, I am trying to visualize what we should do with it. I just want to make sure
that you are indeed trying to introduce paging between HTTP layer and container layer, I am
okay with that, but we could effectively return the whole data since we are sure that web
front will read until it reaches the count number of keys. So we can propagate that to container
and we can return the whole data set, with of course a max cap on how many keys can be read
in a single call.
https://issues.apache.org/jira/secure/attachment/12799549/ozone_user_v0.pdf from page:16 we
discuss the Rest Protocol.
In other words, we should just honor the prefix, start and count. That is return keys that
match a prefix, go to the start key under that prefix and return up to count or less number
of keys.


* {{ListKeyResult.java:}}
  private static final Logger LOG =
      LoggerFactory.getLogger(ContainerMapping.class); ==> ListKeyResult.class

> Ozone: Implement listKey function for KeyManager
> ------------------------------------------------
>
>                 Key: HDFS-11569
>                 URL: https://issues.apache.org/jira/browse/HDFS-11569
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>         Attachments: HDFS-11569-HDFS-7240.001.patch, HDFS-11569-HDFS-7240.002.patch,
HDFS-11569-HDFS-7240.003.patch, HDFS-11569-HDFS-7240.004.patch, HDFS-11569-HDFS-7240.005.patch
>
>
> List keys by prefix from a container. This will need to support pagination for the purpose
of small object support. So the listKey function returns something like ListKeyResult, client
can iterate the object to get pagination results.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message