hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Bockelman <bbock...@cse.unl.edu>
Subject Re: concurrent write [ is it defined and consistent ? ]
Date Tue, 01 Dec 2009 14:30:32 GMT
Hey Krishna,

This type of functionality is not available in HDFS, unfortunately.

The upcoming 0.21.x release will be the first to support appends after the file is closed
(still single-writer though).  I'd speculate that this lays out some ground work that is necessary
for multi-writer atomic appends, but I don't even know if atomic appends are even on the roadmap.

Brian

On Dec 1, 2009, at 8:27 AM, Krishna Kumar wrote:

> Ok , I was just trying to search some record append atomic operation [
> which happens in GFS ? ] . Is there anything  similar in HDFS ?
> 
> Thanks and Best Regards,
> Krishna Kumar
> Senior Storage Engineer 
> Why do we have to die? If we had to die, and everything is gone after
> that, then nothing else matters on this earth - everything is temporary,
> at least relative to me.
> 
> 
> 
> -----Original Message-----
> From: Brian Bockelman [mailto:bbockelm@cse.unl.edu] 
> Sent: Tuesday, December 01, 2009 7:49 PM
> To: common-user@hadoop.apache.org
> Subject: Re: concurrent write [ is it defined and consistent ? ]
> 
> I double checked, and it appears that the answer you are looking for is
> in a funny section ("Data replication"):
> 
> "Files in HDFS are write-once and have strictly one writer at any time.
> "
> "There is a plan to support appending-writes to files in the future."
> 
> So, there are no concurrent writes, but there are future plans for
> appends (currently in trunk).
> 
> Brian
> 
> On Dec 1, 2009, at 8:16 AM, Krishna Kumar wrote:
> 
>> It doesn't give answer to my query .
>> 
>> Thanks and Best Regards,
>> Krishna Kumar
>> Senior Storage Engineer 
>> Why do we have to die? If we had to die, and everything is gone after
>> that, then nothing else matters on this earth - everything is
> temporary,
>> at least relative to me.
>> 
>> 
>> -----Original Message-----
>> From: Brian Bockelman [mailto:bbockelm@cse.unl.edu] 
>> Sent: Tuesday, December 01, 2009 7:32 PM
>> To: common-user@hadoop.apache.org
>> Subject: Re: concurrent write [ is it defined and consistent ? ]
>> 
>> Hey Krishna,
>> 
>> You might benefit from the HDFS design document:
>> 
>> http://hadoop.apache.org/common/docs/current/hdfs_design.html
>> 
>> Let us know if it is missing important information.
>> 
>> Brian
>> 
>> On Dec 1, 2009, at 7:59 AM, Krishna Kumar wrote:
>> 
>>> Dear all,
>>> 
>>> 
>>> 
>>> Can anybody tell me how concurrent write happens in HDFS ?
>>> 
>>> 
>>> 
>>> Thanks and Best Regards,
>>> 
>>> Krishna Kumar
>>> 
>>> Senior Storage Engineer 
>>> 
>>> Why do we have to die? If we had to die, and everything is gone after
>>> that, then nothing else matters on this earth - everything is
>> temporary,
>>> at least relative to me.
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> DISCLAIMER: 
>>> 
>> 
> ------------------------------------------------------------------------
>> ----------------------------------------------- 
>>> The contents of this e-mail and any attachment(s) are confidential
> and
>>> intended 
>>> for the named recipient(s) only.  
>>> It shall not attach any liability on the originator or NECHCL or its 
>>> affiliates. Any views or opinions presented in  
>>> this email are solely those of the author and may not necessarily
>> reflect the 
>>> opinions of NECHCL or its affiliates.  
>>> Any form of reproduction, dissemination, copying, disclosure,
>> modification, 
>>> distribution and / or publication of  
>>> this message without the prior written consent of the author of this
>> e-mail is 
>>> strictly prohibited. If you have  
>>> received this email in error please delete it and notify the sender 
>>> immediately. . 
>>> 
>> 
> ------------------------------------------------------------------------
>> -----------------------------------------------
>> 
>> 
>> 
>> 
>> DISCLAIMER: 
>> 
> ------------------------------------------------------------------------
> ----------------------------------------------- 
>> The contents of this e-mail and any attachment(s) are confidential and
>> intended 
>> for the named recipient(s) only.  
>> It shall not attach any liability on the originator or NECHCL or its 
>> affiliates. Any views or opinions presented in  
>> this email are solely those of the author and may not necessarily
> reflect the 
>> opinions of NECHCL or its affiliates.  
>> Any form of reproduction, dissemination, copying, disclosure,
> modification, 
>> distribution and / or publication of  
>> this message without the prior written consent of the author of this
> e-mail is 
>> strictly prohibited. If you have  
>> received this email in error please delete it and notify the sender 
>> immediately. . 
>> 
> ------------------------------------------------------------------------
> -----------------------------------------------
> 
> 
> 
> 
> DISCLAIMER: 
> -----------------------------------------------------------------------------------------------------------------------

> The contents of this e-mail and any attachment(s) are confidential and
> intended 
> for the named recipient(s) only.  
> It shall not attach any liability on the originator or NECHCL or its 
> affiliates. Any views or opinions presented in  
> this email are solely those of the author and may not necessarily reflect the 
> opinions of NECHCL or its affiliates.  
> Any form of reproduction, dissemination, copying, disclosure, modification, 
> distribution and / or publication of  
> this message without the prior written consent of the author of this e-mail is 
> strictly prohibited. If you have  
> received this email in error please delete it and notify the sender 
> immediately. . 
> -----------------------------------------------------------------------------------------------------------------------


Mime
View raw message