Return-Path: X-Original-To: apmail-hadoop-common-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-common-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id CE4DD18B10 for ; Thu, 5 Nov 2015 16:38:37 +0000 (UTC) Received: (qmail 11904 invoked by uid 500); 5 Nov 2015 16:38:28 -0000 Delivered-To: apmail-hadoop-common-issues-archive@hadoop.apache.org Received: (qmail 11852 invoked by uid 500); 5 Nov 2015 16:38:28 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-issues@hadoop.apache.org Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 11839 invoked by uid 99); 5 Nov 2015 16:38:28 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Nov 2015 16:38:27 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id BFBF62C1F65 for ; Thu, 5 Nov 2015 16:38:27 +0000 (UTC) Date: Thu, 5 Nov 2015 16:38:27 +0000 (UTC) From: "Aaron Peterson (JIRA)" To: common-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HADOOP-11687) Ignore x-emc-* headers when copying an Amazon S3 object MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-11687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14991941#comment-14991941 ] Aaron Peterson commented on HADOOP-11687: ----------------------------------------- Implementing a different cloning method seems to alleviate the COPY issue: 1092c1092 < final ObjectMetadata dstom = srcom.clone(); --- > ObjectMetadata dstom = copyOfObjMetadata(srcom); 1197a1198,1224 > // Constructs copy of object metadata > private ObjectMetadata copyOfObjMetadata(ObjectMetadata source) { > ObjectMetadata ret = new ObjectMetadata(); > ret.setCacheControl(source.getCacheControl()); > ret.setContentDisposition(source.getContentDisposition()); > ret.setContentEncoding(source.getContentEncoding()); > ret.setContentLength(source.getContentLength()); > ret.setContentMD5(source.getContentMD5()); > ret.setContentType(source.getContentType()); > ret.setExpirationTime(source.getExpirationTime()); > ret.setExpirationTimeRuleId(source.getExpirationTimeRuleId()); > ret.setHttpExpiresDate(source.getHttpExpiresDate()); > ret.setLastModified(source.getLastModified()); > ret.setOngoingRestore(source.getOngoingRestore()); > ret.setRestoreExpirationTime(source.getRestoreExpirationTime()); > ret.setSSEAlgorithm(source.getSSEAlgorithm()); > ret.setSSECustomerAlgorithm(source.getSSECustomerAlgorithm()); > ret.setSSECustomerKeyMd5(source.getSSECustomerKeyMd5()); > if (source.getUserMetadata().isEmpty() != Boolean.TRUE) { > java.util.Map smd = source.getUserMetadata(); > for (String key : smd.keySet()) { > ret.addUserMetadata(key, smd.get(key)); > } > } > return ret; > } > > Ignore x-emc-* headers when copying an Amazon S3 object > ------------------------------------------------------- > > Key: HADOOP-11687 > URL: https://issues.apache.org/jira/browse/HADOOP-11687 > Project: Hadoop Common > Issue Type: Improvement > Components: fs/s3 > Affects Versions: 2.7.0 > Reporter: Denis Jannot > > The EMC ViPR/ECS object storage platform uses proprietary headers starting by x-emc-* (like Amazon does with x-amz-*). > Headers starting by x-emc-* should be included in the signature computation, but it's not done by the Amazon S3 Java SDK (it's done by the EMC S3 SDK). > When s3a copy an object it copies all the headers, but when the object includes x-emc-* headers, it generates a signature mismatch. > Removing the x-emc-* headers from the copy would allow s3a to be compatible with the EMC ViPR/ECS object storage platform. > Removing the x-* which aren't x-amz-* headers from the copy would allow s3a to be compatible with any object storage platform which is using proprietary headers -- This message was sent by Atlassian JIRA (v6.3.4#6332)