hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharat Viswanadham (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDDS-517) Implement HeadObject REST endpoint
Date Tue, 09 Oct 2018 20:51:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644069#comment-16644069
] 

Bharat Viswanadham edited comment on HDDS-517 at 10/9/18 8:50 PM:
------------------------------------------------------------------

Hi [~GeLiXin]

Thank You for the updated patch.

Few comments:
 # This patch needs to be rebased on top of trunk.
 # The setting of x-amz-request-id is not required, as this is handled as a common thing for
all request as part of HDDS-585
 # Why do we need this check .header("Content-Length", body == null ? 0 : length), and also
why do we need OutputStream for HEADObject? (Is this added when range comes in to picture)
 # During my testing (I have done some changes to make code compile in my local), I see a
slight change in the output from s3 and our S3Gateway. Few observations is content type and
content length returned for us is zero, it should be the key length. For no key, now we are
returning 404.
 # And also I think no need to add x-amz-version-id by default, when I have copied a key
and see head-object, I don't see version id in default response. (This one I have checked,
after setting up mitmproxy)

 
{code:java}
S3 Gateway:
HW13865:hadoop bviswanadham$ aws s3api --endpoint-url=http://localhost:9878/volume1 head-object
--bucket bucket1 --key dt
{
    "ContentType": "text/plain", 
    "ContentLength": 0, 
    "Expires": "Tue, 09 Oct 2018 20:38:09 GMT", 
    "ETag": "1539117326439", 
    "CacheControl": "no-cache", 
    "Metadata": {}
}
Amazon S3
HW13865:hadoop bviswanadham$ aws s3api --no-verify-ssl head-object --bucket my-bucket-09-09
--key tmp/dt --range 124bytes
/usr/local/aws/lib/python2.7/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning:
Unverified HTTPS request is being made. Adding certificate verification is strongly advised.
See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
{
    "AcceptRanges": "bytes", 
    "ContentType": "binary/octet-stream", 
    "LastModified": "Tue, 09 Oct 2018 20:05:13 GMT", 
    "ContentLength": 5, 
    "ETag": "\"a9564ebc3289b7a14551baf8ad5ec60a\"", 
    "Metadata": {}
}
{code}
 


was (Author: bharatviswa):
Hi [~GeLiXin]

Thank You for the updated patch.

Few comments:
 # This patch needs to be rebased on top of trunk.
 # The setting of x-amz-request-id is not required, as this is handled as a common thing for
all request as part of HDDS-585
 # Why do we need this check .header("Content-Length", body == null ? 0 : length), and also
why do we need OutputStream for HEADObject? (Is this added when range comes in to picture)
 # During my testing (I have done some changes to make code compile in my local), I see a
slight change in the output from s3 and our S3Gateway. Few observations is content type and
content length returned for us is zero, it should be the key length. For no key, now we are
returning 404.

 
{code:java}
S3 Gateway:
HW13865:hadoop bviswanadham$ aws s3api --endpoint-url=http://localhost:9878/volume1 head-object
--bucket bucket1 --key dt
{
    "ContentType": "text/plain", 
    "ContentLength": 0, 
    "Expires": "Tue, 09 Oct 2018 20:38:09 GMT", 
    "ETag": "1539117326439", 
    "CacheControl": "no-cache", 
    "Metadata": {}
}
Amazon S3
HW13865:hadoop bviswanadham$ aws s3api --no-verify-ssl head-object --bucket my-bucket-09-09
--key tmp/dt --range 124bytes
/usr/local/aws/lib/python2.7/site-packages/urllib3/connectionpool.py:857: InsecureRequestWarning:
Unverified HTTPS request is being made. Adding certificate verification is strongly advised.
See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings
  InsecureRequestWarning)
{
    "AcceptRanges": "bytes", 
    "ContentType": "binary/octet-stream", 
    "LastModified": "Tue, 09 Oct 2018 20:05:13 GMT", 
    "ContentLength": 5, 
    "ETag": "\"a9564ebc3289b7a14551baf8ad5ec60a\"", 
    "Metadata": {}
}
{code}
 

> Implement HeadObject REST endpoint
> ----------------------------------
>
>                 Key: HDDS-517
>                 URL: https://issues.apache.org/jira/browse/HDDS-517
>             Project: Hadoop Distributed Data Store
>          Issue Type: Sub-task
>            Reporter: Elek, Marton
>            Assignee: LiXin Ge
>            Priority: Major
>              Labels: newbie
>         Attachments: HDDS-517.000.patch, HDDS-517.001.patch
>
>
> The HEAD operation retrieves metadata from an object without returning the object itself.
This operation is useful if you are interested only in an object's metadata. To use HEAD,
you must have READ access to the object.
> Steps:
>  1. Look up the volume
>  2. Read the key and return to the user.
> The AWS reference is here:
> https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectHEAD.html
> We have a simple version of this call in HDDS-444 but without Range support.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message