hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ossi <los...@gmail.com>
Subject lost data with 1 failed datanode and replication factor 3 in 6 node cluster
Date Fri, 21 Oct 2011 09:26:04 GMT
hi,

We managed to lost data when 1 datanode broke down in a cluster of 6
datanodes with
replication factor 3.

As far as I know, that shouldn't happen, since each blocks should have 1
copy in
3 different hosts. So, loosing even 2 nodes should be fine.

Earlier we did some tests with replication factor 2, but reverted from that:
   88  2011-10-12 06:46:49 hadoop dfs -setrep -w 2 -R /
  148  2011-10-12 10:22:09 hadoop dfs -setrep -w 3 -R /

The lost data was generated after replication factor was set back to 3.
And even if replication factor would have been 2, data shouldn't have been
lost, right?

We wonder how that is possible and in what situations that could happen?


br, Ossi

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message