hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-11951) Ozone: Handle potential inconsistent states while listing keys
Date Thu, 08 Jun 2017 10:14:18 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16042506#comment-16042506
] 

Weiwei Yang edited comment on HDFS-11951 at 6/8/17 10:13 AM:
-------------------------------------------------------------

Hi [~linyiqun]

I think if it all happens on server side, this should not happen like the reason you mentioned.
But I agreed with Anu because if this is a client code which implements pagination list via
rest call, this could happen, like following

# A client send rest call to ozone web layer {{BucketHandler}} and that calls listBuckets
with 0 ~ 100 count. This is the first page.
# Client gets the result back, it returns key1 to key100.
# At this point, someone deletes key100.
# Based on the result, client uses key100 as the startKey for next page request, sending another
rest call to ozone server. This will fail because key100 is gone.

Let me know if this makes sense. Thank you.


was (Author: cheersyang):
Hi [~linyiqun]

I think if it all happens on server side, this should not happen like the reason you mentioned.
But I agreed with Anu because if this is a client code which implements pagination list via
rest call, this could happen, like following

1. A client send rest call to ozone web layer {{BucketHandler}} and that calls listBuckets
with 0 ~ 100 count. This is the first page.
2. Client gets the result back, it returns key1 to key100.
3. At this point, someone deletes key100.
4. Based on the result, client uses key100 as the startKey for next page request, sending
another rest call to ozone server. This will fail because key100 is gone.

Let me know if this makes sense. Thank you.

> Ozone: Handle potential inconsistent states while listing keys
> --------------------------------------------------------------
>
>                 Key: HDFS-11951
>                 URL: https://issues.apache.org/jira/browse/HDFS-11951
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>    Affects Versions: HDFS-7240
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>
> In the discussions in HDFS-11779, there might be a corner case we running into the situation
that some key are deleted while we are listing them. For example
> 1. A list call is made say from key1 to key100 is returned.
> 2. Someone deletes the key100
> 3. A list continue call is made with startKey=key100.
> We need to investigate this case and make sure this is properly handled in ozone. See
more discussion from [here | https://issues.apache.org/jira/browse/HDFS-11779?focusedCommentId=16042356&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16042356].



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message