From common-issues-return-8897-apmail-hadoop-common-issues-archive=hadoop.apache.org@hadoop.apache.org Mon Jun 07 17:12:20 2010 Return-Path: Delivered-To: apmail-hadoop-common-issues-archive@minotaur.apache.org Received: (qmail 30950 invoked from network); 7 Jun 2010 17:12:20 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 7 Jun 2010 17:12:20 -0000 Received: (qmail 62840 invoked by uid 500); 7 Jun 2010 17:12:20 -0000 Delivered-To: apmail-hadoop-common-issues-archive@hadoop.apache.org Received: (qmail 62787 invoked by uid 500); 7 Jun 2010 17:12:20 -0000 Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: common-issues@hadoop.apache.org Delivered-To: mailing list common-issues@hadoop.apache.org Received: (qmail 62779 invoked by uid 99); 7 Jun 2010 17:12:20 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Jun 2010 17:12:20 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.22] (HELO thor.apache.org) (140.211.11.22) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Jun 2010 17:12:17 +0000 Received: from thor (localhost [127.0.0.1]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id o57HBuGF014972 for ; Mon, 7 Jun 2010 17:11:56 GMT Message-ID: <4336412.4871275930716074.JavaMail.jira@thor> Date: Mon, 7 Jun 2010 13:11:56 -0400 (EDT) From: "Hairong Kuang (JIRA)" To: common-issues@hadoop.apache.org Subject: [jira] Commented: (HADOOP-5438) Merge FileSystem.create and FileSystem.append MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HADOOP-5438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12876317#action_12876317 ] Hairong Kuang commented on HADOOP-5438: --------------------------------------- > Here is not about append and create. it is also create and overwrite. Reverting the patch does not remove the "overwrite" functionality. If we only need to support create and overwrite, a boolean flag is good enough as we have pre this patch. I do not see the need of introducing an enum. Yongqiang, could you please explain if there is anything beyond which makes you against the idea of reverting this patch? > Merge FileSystem.create and FileSystem.append > --------------------------------------------- > > Key: HADOOP-5438 > URL: https://issues.apache.org/jira/browse/HADOOP-5438 > Project: Hadoop Common > Issue Type: Improvement > Components: fs > Reporter: He Yongqiang > Assignee: He Yongqiang > Fix For: 0.21.0 > > Attachments: Hadoop-5438(2009-04-06).patch, Hadoop-5438-2009-03-30.patch, Hadoop-5438-2009-03-31-2.patch, Hadoop-5438-2009-03-31.patch, Hadoop-5438-2009-05-10.patch, Hadoop-5438-2009-05-15.patch, Hadoop-5438-2009-05-19.patch, Hadoop-5438-2009-05-5.patch > > > Currently, when a user wants to modify a file, the user first calls exists() to know if this file is already there. And then uses create() or append() according to whether the file exists or not. > the code looks like: > {code} > FSDataOutputStream out_1 = null; > if (fs.exists(path_1)) > out_1 = fs.append(path_1); > else > out_1 = fs.create(path_1); > {code} > . On the performace side,It involes two RPCs. On the easy-of-use side, it is not very convient in contrast to the traditional open interface. > It will more complicate if there is a overwrite parameter specified. I donot know whether there is a bug about 'overwrite' in 0.19, some times it takes a long time for overwrite creates to reture. So i make the write file code with overwrite param works like: > {code} > boolean exists = fs.exists(name); > if (overwrite) { > if (exists) > fs.delete(name, true); > this.out = fs.create(name, overwrite, bufferSize, replication, > blockSize, progress); > this.currentRowID = 0; > } else { > if (!exists) > this.out = fs.create(name, overwrite, bufferSize, > replication, blockSize, progress); > else > this.out = fs.append(name, bufferSize, progress); > {code} > Some code statements there are really redundant and not needed, especialy with the delete(). But without deleting first, the overwrite takes a long time to reture. > BTW, i will create another issue about the overwrite problem. If it is not a bug at all or a duplicate, someone please close it. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.