hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thanh Do <than...@cs.wisc.edu>
Subject Re: silent data loss during append
Date Fri, 15 Apr 2011 15:02:05 GMT
I am using cloudera's distribution version:
hadoop-0.20.2+738.

On Thu, Apr 14, 2011 at 6:23 PM, Ted Dunning <tdunning@maprtech.com> wrote:

> What version are you using?
>
>
> On Thu, Apr 14, 2011 at 3:55 PM, Thanh Do <thanhdo@cs.wisc.edu> wrote:
>
>> Hi all,
>>
>> I have recently seen silent data loss in our system.
>> Here is the case:
>>
>> 1. client appends to some block
>> 2. for some reason, commitBlockSynchronization
>>     returns successfully with synclist = [] (i.e empty)
>> 3. in the client code, NO exception is thrown, and
>>    client appends successfully.
>> 4. However, the block replicas are then removed from
>>    datanodes, causing data loss.
>>
>> Have any one seen this before?
>> Is this behavior by design or a bug?
>>
>> Many thanks,
>> Thanh
>>
>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message