hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharat Viswanadham (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-516) Implement CopyObject REST endpoint
Date Tue, 16 Oct 2018 21:19:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652459#comment-16652459
] 

Bharat Viswanadham commented on HDDS-516:
-----------------------------------------

Ran robot tests with S3Gateway endpoint and AWS endpoint.

 
{code:java}
HW13865:smoketest bviswanadham$ ./test.sh --env ozones3 s3
-------------------------------------------------
Executing test(s): [s3]
 
  Cluster type:      ozones3
  Compose file:      /Users/bviswanadham/workspace/open-hadoop/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/smoketest/../compose/ozones3/docker-compose.yaml
  Output dir:        /Users/bviswanadham/workspace/open-hadoop/hadoop/hadoop-ozone/dist/target/ozone-0.4.0-SNAPSHOT/smoketest/result
  Command to rerun:  ./test.sh --keep --env ozones3 s3
-------------------------------------------------
Removing network ozones3_default
WARNING: Network ozones3_default not found.
Creating network "ozones3_default" with the default driver
Creating ozones3_s3g_1 ... 
Creating ozones3_datanode_1 ... 
Creating ozones3_ozoneManager_1 ... 
Creating ozones3_scm_1 ... 
Creating ozones3_s3g_1
Creating ozones3_scm_1
Creating ozones3_ozoneManager_1
Creating ozones3_ozoneManager_1 ... done
Waiting 30s for cluster start up...
==============================================================================
S3                                                          
                 
==============================================================================
S3.Awss3 :: S3 gateway test with aws cli                                  
   
==============================================================================
File upload and directory list                                       
| PASS |
------------------------------------------------------------------------------
S3.Awss3 :: S3 gateway test with aws cli                              | PASS
|
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
S3.Buckethead :: S3 gateway test with aws cli                             
   
==============================================================================
Head Bucket not existent                                           
  | PASS |
------------------------------------------------------------------------------
S3.Buckethead :: S3 gateway test with aws cli                         | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
S3.Bucketlist :: S3 gateway test with aws cli                             
   
==============================================================================
List buckets                                                   
      | PASS |
------------------------------------------------------------------------------
S3.Bucketlist :: S3 gateway test with aws cli                         | PASS |
1 critical test, 1 passed, 0 failed
1 test total, 1 passed, 0 failed
==============================================================================
S3.Objectcopy :: S3 gateway test with aws cli                             
   
==============================================================================
Copy Object Happy Scenario                                           
| PASS |
------------------------------------------------------------------------------
Copy Object Where Bucket is not available                             | PASS
|
------------------------------------------------------------------------------
Copy Object Where both source and dest are same                       | PASS |
------------------------------------------------------------------------------
Copy Object Where Key not available                                   | PASS
|
------------------------------------------------------------------------------
S3.Objectcopy :: S3 gateway test with aws cli                         | PASS |
4 critical tests, 4 passed, 0 failed
4 tests total, 4 passed, 0 failed
==============================================================================
S3.Objectdelete :: S3 gateway test with aws cli                             
 
==============================================================================
Delete file with s3api                                             
  | PASS |
------------------------------------------------------------------------------
Delete file with s3api, file doesn't exist                            | PASS
|
------------------------------------------------------------------------------
Delete dir with s3api                                              
  | PASS |
------------------------------------------------------------------------------
Delete file with s3api, file doesn't exist, prefix of a real file     | PASS |
------------------------------------------------------------------------------
Delete file with s3api, bucket doesn't exist                          | PASS
|
------------------------------------------------------------------------------
S3.Objectdelete :: S3 gateway test with aws cli                       | PASS |
5 critical tests, 5 passed, 0 failed
5 tests total, 5 passed, 0 failed
==============================================================================
S3.Objectputget :: S3 gateway test with aws cli                             
 
==============================================================================
Put object to s3                                                 
    | PASS |
------------------------------------------------------------------------------
Get object from s3                                               
    | PASS |
------------------------------------------------------------------------------
S3.Objectputget :: S3 gateway test with aws cli                       | PASS |
2 critical tests, 2 passed, 0 failed
2 tests total, 2 passed, 0 failed
==============================================================================
S3                                                           
        | PASS |
14 critical tests, 14 passed, 0 failed
14 tests total, 14 passed, 0 failed
{code}
 

 

 

 
{code:java}
HW13865:smoketest bviswanadham$ robot -v OZONE_S3_SET_CREDENTIALS:false -v BUCKET:b1-9090
-v DESTBUCKET:b1-9091 -v ENDPOINT_URL:https://s3.us-east-1.amazonaws.com s3/objectcopy.robot

==============================================================================
Objectcopy :: S3 gateway test with aws cli                                
   
==============================================================================
Copy Object Happy Scenario                                           
| PASS |
------------------------------------------------------------------------------
Copy Object Where Bucket is not available                             | PASS
|
------------------------------------------------------------------------------
Copy Object Where both source and dest are same                       | PASS |
------------------------------------------------------------------------------
Copy Object Where Key not available                                   | PASS
|
------------------------------------------------------------------------------
Objectcopy :: S3 gateway test with aws cli                            | PASS
|
4 critical tests, 4 passed, 0 failed
4 tests total, 4 passed, 0 failed
==============================================================================
{code}
 

 

> Implement CopyObject REST endpoint
> ----------------------------------
>
>                 Key: HDDS-516
>                 URL: https://issues.apache.org/jira/browse/HDDS-516
>             Project: Hadoop Distributed Data Store
>          Issue Type: Sub-task
>            Reporter: Elek, Marton
>            Assignee: Bharat Viswanadham
>            Priority: Major
>              Labels: newbie
>         Attachments: HDDS-516.01.patch, HDDS-516.03.patch, HDDS-516.04.patch
>
>
> The Copy object is a simple call to Ozone Manager.  This API can only be done after the
PUT OBJECT Call.
> This implementation of the PUT operation creates a copy of an object that is already
stored in Amazon S3. A PUT copy operation is the same as performing a GET and then a PUT.
Adding the request header, x-amz-copy-source, makes the PUT operation copy the source object
into the destination bucket.
> If the Put Object call has this header, then Put Object call will issue a rename. 
> Work Items or JIRAs
> Detect the presence of the extra header - x-amz-copy-source
> Make sure that destination bucket exists.
> The AWS reference is here:
> https://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html
> (This jira is marked as newbie as it requires only basic Ozone knowledge. If somebody
would be interested, I can be more specific, explain what we need or help).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message