hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bharat Viswanadham (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDDS-739) Support MultiDeleteRequest without XML namespace
Date Sat, 27 Oct 2018 16:31:00 GMT

    [ https://issues.apache.org/jira/browse/HDDS-739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16666159#comment-16666159
] 

Bharat Viswanadham commented on HDDS-739:
-----------------------------------------

+1 LGTM. Thank You [~elek]  for fixing this issue.

I have tried curl requests for multi delete with and without namespace, it worked fine.

I think it would be nice to have a robot tests for curl also for our s3 endpoint. I think
we can do in separate Jira as we need to add for all s3 API requests. I will open a new Jira
for this.

I will commit this shortly.

 

> Support MultiDeleteRequest without XML namespace
> ------------------------------------------------
>
>                 Key: HDDS-739
>                 URL: https://issues.apache.org/jira/browse/HDDS-739
>             Project: Hadoop Distributed Data Store
>          Issue Type: Sub-task
>          Components: S3
>            Reporter: Elek, Marton
>            Assignee: Elek, Marton
>            Priority: Major
>         Attachments: HDDS-739.001.patch
>
>
> When I run the IContract* unit tests from s3a adapter to test our s3 gateway I found
that some of the unit tests (such as {color:#000000}ITestS3AContractGetFileStatus{color}.testListStatusEmptyDirectory)
fails with org.apache.hadoop.fs.s3a.AWSBadRequestException.
> Checking the request/response I found that the multi delete request (POST to the bucket
address with ending with ?delete) fails with HTTP 400 and without any error in the logs.
> The multi delete endpoint was tested with the robot tests and it worked well with aws
s3 cli:
> {code:java}
> aws s3api --endpoint http://localhost:9878 delete-objects --bucket buckettest --delete
'Objects=[{Key=multidelete/f1},{Key=multidelete/f2},{Key=multidelete/f4}]'{code}
> After some more investigation I found that the XML namespace was missing from the request:
> The request was something like this:
> {code:java}
> <Delete><Object><Key>multidelete/f1</Key></Object><Object><Key>multidelete/f2</Key></Object><Object><Key>multidelete/f4</Key></Object></Delete>{code}
> Instead of this:
> {code:java}
> <Delete xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Object><Key>multidelete/f1</Key></Object><Object><Key>multidelete/f2</Key></Object><Object><Key>multidelete/f4</Key></Object></Delete>{code}
> I believe the right approach is to add the namespace to request as it's documented by
the AWS documentation but it seems s3a adapter doesn't do it and s3 accepts it.
> We need a custom jaxb unmarshaller logic for the multi delete requests to accept requests
both with and without namespaces.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message