hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hong Tang <ht...@yahoo-inc.com>
Subject Re: concurrent write [ is it defined and consistent ? ]
Date Tue, 01 Dec 2009 18:56:21 GMT
Atomic append is useful in contexts like transactional logging and  
producer-consumer workload sharing systems. Such usage cases are not  
covered by map-reduce programming model.

On Dec 1, 2009, at 8:02 AM, Owen O'Malley wrote:

> On Dec 1, 2009, at 6:30 AM, Brian Bockelman wrote:
>> The upcoming 0.21.x release will be the first to support appends  
>> after the file is closed (still single-writer though).  I'd  
>> speculate that this lays out some ground work that is necessary for  
>> multi-writer atomic appends, but I don't even know if atomic  
>> appends are even on the roadmap.
> The short answer is that I don't think anyone has even thought  
> deeply about it. I haven't heard anyone talking about doing it  
> anytime soon.
> My personal inclination is that atomic append does very bad things  
> to both the design of the file system and the user interface to the  
> file system. Clearly they added atomic append to GFS before they had  
> MapReduce. It seems like most applications would be better served by  
> implementing in MapReduce rather than using atomic append anyways...
> -- Owen

View raw message