Mailing-List: contact core-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of stuart.white1@gmail.com
 designates 209.85.217.215 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:date:message-id:subject:from:to:content-type
         :content-transfer-encoding;
        b=iVEERv71L9ED4CudheuJql5hUBfhV6UHpjj/b0PXes8y8nNrcFu6sYTttN5j+umHu6
         mWX6kQ9jRUQUzXpSx5W6vc+sQOOwVudTdJQkYzMAC1hqAEp9giogMTsAn7XkId4/BKkX
         6yyWEMZhAYZgUzEVsX5DBH0MnBEXZpiG9uA+4=
MIME-Version: 1.0
Date: Tue, 23 Jun 2009 20:18:39 -0500
Message-ID: <4af5cd780906231818s3675a317r64187ad67cc33b32@mail.gmail.com>
Subject: Does balancer ensure a file's replication is satisfied?
From: Stuart White <stuart.white1@gmail.com>
To: core-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

In my Hadoop cluster, I've had several drives fail lately (and they've
been replaced).  Each time a new empty drive is placed in the cluster,
I run the balancer.

I understand that the balancer will redistribute the load of file
blocks across the nodes.

My question is: will balancer also look at the desired replication of
a file, and if the actual replication of a file is less than the
desired (because the file had blocks stored on the lost drive), will
balancer re-replicate those lost blocks?

If not, is there another tool that will ensure the desired replication
factor of files is satisfied?

If this functionality doesn't exist, I'm concerned that I'm slowly,
silently losing my files as I replace drives, and I may not even
realize it.

Thoughts?