Mailing-List: contact common-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of lossil@gmail.com designates
 209.85.220.176 as permitted sender)
MIME-Version: 1.0
Date: Fri, 21 Oct 2011 12:26:04 +0300
Message-ID: 
 <CAOwicohp85Dkv9+UQHBvvPVfN=va0QKoDt7gpDZF8XkZcbf1eg@mail.gmail.com>
Subject: lost data with 1 failed datanode and replication factor 3 in 6 node
 cluster
From: Ossi <lossil@gmail.com>
To: common-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=bcaec5015d518aaa3104afcba939

--bcaec5015d518aaa3104afcba939
Content-Type: text/plain; charset=ISO-8859-1

hi,

We managed to lost data when 1 datanode broke down in a cluster of 6
datanodes with
replication factor 3.

As far as I know, that shouldn't happen, since each blocks should have 1
copy in
3 different hosts. So, loosing even 2 nodes should be fine.

Earlier we did some tests with replication factor 2, but reverted from that:
   88  2011-10-12 06:46:49 hadoop dfs -setrep -w 2 -R /
  148  2011-10-12 10:22:09 hadoop dfs -setrep -w 3 -R /

The lost data was generated after replication factor was set back to 3.
And even if replication factor would have been 2, data shouldn't have been
lost, right?

We wonder how that is possible and in what situations that could happen?


br, Ossi

--bcaec5015d518aaa3104afcba939--