Mailing-List: contact hdfs-user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of dhruba@gmail.com designates
 209.85.211.191 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=J/o18F/4VsalyWmxjDP1iSGIRR0xO5Z0Uhca+/4UVMtEwDRObSRkxAufS+Did26AOz
         BLqngLkqOWMJEDmb0CH5ncmwbQELIY0IETt5oFGp//SzT3MRc/8bbmp1P08Zkp+Ttw3t
         T9QiP5zuHzlkOzax2ltni/xOf70CHQLGNLDLM=
MIME-Version: 1.0
In-Reply-To: <C6BD86F9.15A%awittenauer@linkedin.com>
References: <C6BD86F9.15A%awittenauer@linkedin.com>
Date: Fri, 28 Aug 2009 13:41:24 -0700
Message-ID: <4aa34eb70908281341j4959f21rf2363b4e5816fb58@mail.gmail.com>
Subject: Re: Intelligence of decommission
From: Dhruba Borthakur <dhruba@gmail.com>
To: hdfs-user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=000e0cd6a8082406db047239b54b

--000e0cd6a8082406db047239b54b
Content-Type: text/plain; charset=ISO-8859-1

when a user issues the decommission command, all the blocks that are
currently residing on it are inserted into the to-be-replicated queue. Then
the ReplicationMonitor inside the namenode starts replicating these blocks
(during this period, the replica on the machine being decommissioned is used
for reads, but is not considered a valid replica by the ReplicationMonitor).

hope this helps,
dhruba


On Fri, Aug 28, 2009 at 1:07 PM, Allen Wittenauer
<awittenauer@linkedin.com>wrote:

> Hi.
>
> As I sit here and wait for node decommission to finish, I was wondering
> about the intelligence of the decision making.  [The name nodes, not mine.
> :) ]
>
> Let's say I have the following scenario:
>
> I have two files.  Both files consist of one block with a replication
> factor
> of three.  I decommission two nodes.  File #1 has two of its replicas on
> the
> two nodes I am decommissioning.  File #2 has only one of its replicas on
> one
> of the two nodes I am decommissioning.
>
> Is the block with two replicas on the two nodes I am decommissioning given
> priority?  How does the name node decide which blocks to re-replicate
> first?
>
> Thanks.
>
>

--000e0cd6a8082406db047239b54b
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

when a user issues the decommission command, all the blocks that are curren=
tly residing on it are inserted into the to-be-replicated queue. Then the R=
eplicationMonitor inside the namenode starts replicating these blocks (duri=
ng this period, the replica on the machine being decommissioned is used for=
 reads, but is not considered a valid replica by the ReplicationMonitor).<b=
r>
<br>hope this helps,<br>dhruba<br><br><br><div class=3D"gmail_quote">On Fri=
, Aug 28, 2009 at 1:07 PM, Allen Wittenauer <span dir=3D"ltr">&lt;<a href=
=3D"mailto:awittenauer@linkedin.com">awittenauer@linkedin.com</a>&gt;</span=
> wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"border-left: 1px solid rgb(204, =
204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Hi.<br>
<br>
As I sit here and wait for node decommission to finish, I was wondering<br>
about the intelligence of the decision making. =A0[The name nodes, not mine=
.<br>
:) ]<br>
<br>
Let&#39;s say I have the following scenario:<br>
<br>
I have two files. =A0Both files consist of one block with a replication fac=
tor<br>
of three. =A0I decommission two nodes. =A0File #1 has two of its replicas o=
n the<br>
two nodes I am decommissioning. =A0File #2 has only one of its replicas on =
one<br>
of the two nodes I am decommissioning.<br>
<br>
Is the block with two replicas on the two nodes I am decommissioning given<=
br>
priority? =A0How does the name node decide which blocks to re-replicate fir=
st?<br>
<br>
Thanks.<br>
<br>
</blockquote></div><br>

--000e0cd6a8082406db047239b54b--