hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Anu Engineer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8695) OzoneHandler : Add Bucket REST Interface
Date Fri, 24 Jul 2015 18:39:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14640896#comment-14640896

Anu Engineer commented on HDFS-8695:

[~kanaka] Thanks for your detail review, really appreciate it. I have made modifications to
code and I will soon update the patch.

bq. 1) BucketHandler#createBucket(): parsing remove ACLs is not required, infact should result
in error for unexpected parameter

good catch, I have added a boolean param to getAcls so we can ask getAcls to parse for remove
or not. if remove acls is found when
we don't expect it we will throw an exception.

bq. 2) BucketHandler#upadteBucket(): As per Design specification, right now no plan to support
Bucket quota. But BucketArgs extends VolumeArgs so, OzoneQuota parameter in request can be
invalidated as UnSupportedOperation for now

All data that comes into these handlers come from HTTP headers. Right now, we ignore all headers
that we don't recognize -- (Eventually we will throw for headers that we don't recognize;
something that should be done in BucketProcessTemplate). Since we don't parse that HTTP header
that represents Quota in the bucket path the handler code will never see the quota as an argument.

bq. Also, can you update javadoc to describe current supported operation during update Bucket
so that we can validate the interface contract in implementation

Thanks, I have updated the Java doc.

bq. A new BUCKET_NOT_FOUND error needs to be added in ErrorTable

You are *absolutely* right, please wait until my next patch that adds local storage handling
of buckets. This patch acts as a pass through to underlying layers that actually does the
work. There were a bunch of review comments earlier about not bringing in constants until
they are used. So just following the standard pattern here.

bq. BucketHandler#listBucket : as per design document, I think OZONE_LIST_QUERY_BUCKET should
list the buckets in a volume and supposed to be handled in VolumeHandler#getVolumeInfo(...)
(There is already a TODO to handle this)

I am actually open to suggestions on this. let me explain what is happening here. You have
2 operations on Volumes and buckets. 

* You want to get info about the volume / bucket
* You want to get list of child objects in the volume / bucket

In the case you send in a query like ?info=bucket to a volume object, it interprets it as
it needs a list of all child objects.

if you send the a query like ?info=bucket to a bucket object, it  interprets it as you would
like to learn the meta-data about that bucket.

if you look at the code that is volumeHandler, you will see we call into {{getBucketsInVolume}}
and in the bucketHandler we call into {{getBucketInfoResponse}}. 

[I have a TODO to document these protocols in greater depth]

bq.6) BucketHandler#listBucket: Can you remove javadoc on doProcess() to improve readability
as its already documented in BucketProcessTemplate


bq. If so, BucketHandler.#listBucket: OZONE_LIST_QUERY_BUCKET has to be replaced with OZONE_LIST_QUERY_SERVICE
for getting getBucketInfoResponse()

As I said I need to document the protocol. However this is brief summary, of how the info
key works against objects in ozone.

|| ||?info=service||?info=volume||?info=bucket||?info=key||
|volume|list volumes|info volumes|list buckets| N/A|
|bucket|N/A|N/A|info bucket| list keys|
|Key|N/A|N/A|N/A|info key|

In other words

With reference to volumes :
?info=service - list the volumes owner by a user or if you are an admin for the requested

?info=volume - Metadata about the volume, including this like Quota, who the Owner is etc.etc.

?info=bucket - list of all buckets in a volume.

?info=key - Invalid on Volumes

With reference to buckets :
?info=service - Invalid

?info=volume - Invalid

?info=bucket - metadata about the bucket

?info=key - list of keys

With reference to keys :
?info=service - Invalid

?info=volume - Invalid

?info=bucket - invalid

?info=key - metadata about a specific key

Please let me know if this makes sense or if you would like to see more clarifications.

> OzoneHandler : Add Bucket REST Interface
> ----------------------------------------
>                 Key: HDFS-8695
>                 URL: https://issues.apache.org/jira/browse/HDFS-8695
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: ozone
>            Reporter: Anu Engineer
>            Assignee: Anu Engineer
>         Attachments: hdfs-8695-HDFS-7240.001.patch
> Add Bucket REST interface into Ozone server.

This message was sent by Atlassian JIRA

View raw message