Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: 
 <CA+4kjVuuOpOfzSAg+gxb0VyTUvKxKvkmJNyapHyV2MKzMify+A@mail.gmail.com>
References: 
 <CAEGG_eMC7_YPkvGZBMNxHO5O63cx63fn4wiVW28dFB6TSdCg-Q@mail.gmail.com>
 <CA+4kjVuG_hkcwVkXDFH55xkUW02Wrc5wTfmtg4AchRZh4y9pUg@mail.gmail.com>
 <CAEGG_ePKtLxrdCO7cy0+QvpiZOCYpEtSQ9PDm7Z4ydexDJ+EVg@mail.gmail.com>
 <CA+4kjVuuOpOfzSAg+gxb0VyTUvKxKvkmJNyapHyV2MKzMify+A@mail.gmail.com>
From: Ted Dunning <tdunning@maprtech.com>
Date: Tue, 9 Oct 2012 12:31:05 -0700
Message-ID: 
 <CAND0qzuKpYR58-h9HsLXYChpH2f3NvShCA=e7Dv=K7aAdPV=1A@mail.gmail.com>
Subject: Re: How to change topology
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=0023545303b04560f504cba562ce

--0023545303b04560f504cba562ce
Content-Type: text/plain; charset=ISO-8859-1

On Tue, Oct 9, 2012 at 12:17 PM, Steve Loughran <stevel@hortonworks.com>wrote:

>
>
> On 9 October 2012 16:51, Shinichi Yamashita <shin1.ym@gmail.com> wrote:
>
>> Hi Steve,
>>
>> Thank you for your reply.
>>
>>
>> > no, it's the Namenode and JobTracker that needs to be restarted;
>> > they are the bits that care where the boxes are.
>>
>> I confirmed it in my cluster, and I understood it as follows.
>> First, the resolved node information is recorded in ConcurrentHashMap.
>> Next same node toplogy decision, HashMap data is given priority to over a
>> script.
>> The HashMap data become invalid by restart of NameNode and JobTracker.
>>
>
> That's right. There's been lots of discussion on adding live topology
> resetting.
>

For what it's worth, there is at least one Hadoop derived distribution
which has live topology resetting already.

I can follow up offline with anyone who wants more information.

--0023545303b04560f504cba562ce
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

<br><br><div class=3D"gmail_quote">On Tue, Oct 9, 2012 at 12:17 PM, Steve L=
oughran <span dir=3D"ltr">&lt;<a href=3D"mailto:stevel@hortonworks.com" tar=
get=3D"_blank">stevel@hortonworks.com</a>&gt;</span> wrote:<br><blockquote =
class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid=
;padding-left:1ex">

<br><br><div class=3D"gmail_quote">On 9 October 2012 16:51, Shinichi Yamash=
ita <span dir=3D"ltr">&lt;<a href=3D"mailto:shin1.ym@gmail.com" target=3D"_=
blank">shin1.ym@gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gma=
il_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-lef=
t:1ex">


Hi Steve,<br><br>Thank you for your reply.<div><br><br>&gt; no, it&#39;s th=
e Namenode and JobTracker that needs to be restarted; <br>&gt; they are the=
 bits that care where the boxes are.<br><br></div>I confirmed it in my clus=
ter, and I understood it as follows.<br>


First, the resolved node information is recorded in ConcurrentHashMap.<br>N=
ext same node toplogy decision, HashMap data is given priority to over a sc=
ript.<br>The HashMap data become invalid by restart of NameNode and JobTrac=
ker.<br>


</blockquote><div><br></div><div>That&#39;s right. There&#39;s been lots of=
 discussion on adding live topology resetting.=A0</div></div></blockquote><=
div><br></div><div>For what it&#39;s worth, there is at least one Hadoop de=
rived distribution which has live topology resetting already.</div>

<div><br></div><div>I can follow up offline with anyone who wants more info=
rmation.</div><div><br></div></div>

--0023545303b04560f504cba562ce--