Return-Path: X-Original-To: apmail-hadoop-common-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-common-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0933518DC7 for ; Thu, 6 Aug 2015 18:41:13 +0000 (UTC) Received: (qmail 22074 invoked by uid 500); 6 Aug 2015 18:41:06 -0000 Delivered-To: apmail-hadoop-common-issues-archive@hadoop.apache.org Received: (qmail 22028 invoked by uid 500); 6 Aug 2015 18:41:06 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-issues@hadoop.apache.org Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 22017 invoked by uid 99); 6 Aug 2015 18:41:06 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Aug 2015 18:41:06 +0000 Date: Thu, 6 Aug 2015 18:41:06 +0000 (UTC) From: "Steve Loughran (JIRA)" To: common-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HADOOP-12057) swiftfs rename on partitioned file attempts to consolidate partitions MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HADOOP-12057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660589#comment-14660589 ] Steve Loughran commented on HADOOP-12057: ----------------------------------------- Apparently there's a Sahara patch related to this: https://bugs.launchpad.net/sahara/+bug/1428941 how are they different? > swiftfs rename on partitioned file attempts to consolidate partitions > --------------------------------------------------------------------- > > Key: HADOOP-12057 > URL: https://issues.apache.org/jira/browse/HADOOP-12057 > Project: Hadoop Common > Issue Type: Bug > Components: fs/swift > Reporter: David Dobbins > Assignee: David Dobbins > Attachments: HADOOP-12057-006.patch, HADOOP-12057.patch, HADOOP-12057.patch, HADOOP-12057.patch, HADOOP-12057.patch, HADOOP-12057.patch > > > In the swift filesystem for openstack, a rename operation on a partitioned file uses the swift COPY operation, which attempts to consolidate all of the partitions into a single object. This causes the rename to fail when the total size of all the partitions exceeds the maximum object size for swift. Since partitioned files are primarily created to allow a file to exceed the maximum object size, this bug makes writing to swift extremely unreliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)