hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Owen O'Malley <omal...@apache.org>
Subject Re: concurrent write [ is it defined and consistent ? ]
Date Tue, 01 Dec 2009 16:02:58 GMT

On Dec 1, 2009, at 6:30 AM, Brian Bockelman wrote:

> The upcoming 0.21.x release will be the first to support appends  
> after the file is closed (still single-writer though).  I'd  
> speculate that this lays out some ground work that is necessary for  
> multi-writer atomic appends, but I don't even know if atomic appends  
> are even on the roadmap.

The short answer is that I don't think anyone has even thought deeply  
about it. I haven't heard anyone talking about doing it anytime soon.

My personal inclination is that atomic append does very bad things to  
both the design of the file system and the user interface to the file  
system. Clearly they added atomic append to GFS before they had  
MapReduce. It seems like most applications would be better served by  
implementing in MapReduce rather than using atomic append anyways...

-- Owen

View raw message