Return-Path: Delivered-To: apmail-lucene-hadoop-user-archive@locus.apache.org Received: (qmail 18113 invoked from network); 5 Nov 2007 18:34:47 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 5 Nov 2007 18:34:47 -0000 Received: (qmail 94477 invoked by uid 500); 5 Nov 2007 18:32:45 -0000 Delivered-To: apmail-lucene-hadoop-user-archive@lucene.apache.org Received: (qmail 94449 invoked by uid 500); 5 Nov 2007 18:32:45 -0000 Mailing-List: contact hadoop-user-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: hadoop-user@lucene.apache.org Delivered-To: mailing list hadoop-user@lucene.apache.org Received: (qmail 94440 invoked by uid 99); 5 Nov 2007 18:32:44 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Nov 2007 10:32:44 -0800 X-ASF-Spam-Status: No, hits=1.2 required=10.0 tests=SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [208.75.86.161] (HELO vafer.org) (208.75.86.161) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 05 Nov 2007 18:32:47 +0000 Received: from e179131249.adsl.alicedsl.de ([85.179.131.249]:38378 helo=[192.168.1.2]) by vafer.org with esmtpsa (TLS-1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.62) (envelope-from ) id 1Ip6kN-0003tI-17 for hadoop-user@lucene.apache.org; Mon, 05 Nov 2007 18:32:27 +0000 Mime-Version: 1.0 (Apple Message framework v752.3) In-Reply-To: <84E52AD05F6F884AAFF3344FE4C95991483116@SNV-EXVS08.ds.corp.yahoo.com> References: <84E52AD05F6F884AAFF3344FE4C95991483116@SNV-EXVS08.ds.corp.yahoo.com> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: Content-Transfer-Encoding: 7bit From: Torsten Curdt Subject: Re: rename dir while writing Date: Mon, 5 Nov 2007 19:32:34 +0100 To: hadoop-user@lucene.apache.org X-Mailer: Apple Mail (2.752.3) X-Virus-Checked: Checked by ClamAV on apache.org What about renaming the directory the file is in? ...not the file itself. I would have expected that writing is only "applied" once the file is getting closed. So in theory the final directory would not even have to exist before the file is close. ...but that was just a wild assumption. Anyway - point is: this really needs to be documented somewhere! cheers -- Torsten On 05.11.2007, at 18:58, dhruba Borthakur wrote: > Starting with release 0.15, a file will appear in the namespace as > soon > as it is created. If a writer is writing to a file and another client > renames it, then the original writer will get an IO exception either > when it finished writing to the current block or when it closes the > file. > > Thanks, > dhruba > > -----Original Message----- > From: Torsten Curdt [mailto:tcurdt@apache.org] > Sent: Monday, November 05, 2007 2:05 AM > To: hadoop-user@lucene.apache.org > Subject: rename dir while writing > > Is there anywhere documented the expected behavior of concurrent > changes in the filesystem? As an example: > > Hdfs client C1 is slowly writing to "/path/a/file". Now another hdfs > client C2 renames "/path/a" to /path/b". > > What happens? Will C1 continue to write but the file will be in "/ > path/b" when it closes the file? Will C2 get an exception because C1 > is writing to "/path/a" so it cannot be renamed? And so on... > > I will probably run off an do some test myself ...but is this > documented somewhere? I couldn't find it. > > cheers > -- > Torsten