hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thanh Do <than...@cs.wisc.edu>
Subject silent data loss during append
Date Thu, 14 Apr 2011 22:55:51 GMT
Hi all,

I have recently seen silent data loss in our system.
Here is the case:

1. client appends to some block
2. for some reason, commitBlockSynchronization
    returns successfully with synclist = [] (i.e empty)
3. in the client code, NO exception is thrown, and
   client appends successfully.
4. However, the block replicas are then removed from
   datanodes, causing data loss.

Have any one seen this before?
Is this behavior by design or a bug?

Many thanks,

View raw message