hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "dhruba Borthakur" <dhr...@yahoo-inc.com>
Subject RE: rename dir while writing
Date Tue, 06 Nov 2007 00:24:24 GMT
Renaming the directory shows the same behavior as renaming the file. The
writer will encounter an Exception.  

Thanks,
dhruba

-----Original Message-----
From: Torsten Curdt [mailto:tcurdt@apache.org] 
Sent: Monday, November 05, 2007 10:33 AM
To: hadoop-user@lucene.apache.org
Subject: Re: rename dir while writing

What about renaming the directory the file is in? ...not the file  
itself.

I would have expected that writing is only "applied" once the file is  
getting closed. So in theory the final directory would not even have  
to exist before the file is close. ...but that was just a wild  
assumption.

Anyway - point is: this really needs to be documented somewhere!

cheers
--
Torsten

On 05.11.2007, at 18:58, dhruba Borthakur wrote:

> Starting with release 0.15, a file will appear in the namespace as  
> soon
> as it is created. If a writer is writing to a file and another client
> renames it, then the original writer will get an IO exception either
> when it finished writing to the current block or when it closes the
> file.
>
> Thanks,
> dhruba
>
> -----Original Message-----
> From: Torsten Curdt [mailto:tcurdt@apache.org]
> Sent: Monday, November 05, 2007 2:05 AM
> To: hadoop-user@lucene.apache.org
> Subject: rename dir while writing
>
> Is there anywhere documented the expected behavior of concurrent
> changes in the filesystem? As an example:
>
> Hdfs client C1 is slowly writing to "/path/a/file". Now another hdfs
> client C2 renames "/path/a" to /path/b".
>
> What happens? Will C1 continue to write but the file will be in "/
> path/b" when it closes the file? Will C2 get an exception because C1
> is writing to "/path/a" so it cannot be renamed? And so on...
>
> I will probably run off an do some test myself ...but is this
> documented somewhere? I couldn't find it.
>
> cheers
> --
> Torsten


Mime
View raw message