Return-Path: X-Original-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-mapreduce-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B9E55D70E for ; Wed, 10 Oct 2012 15:20:25 +0000 (UTC) Received: (qmail 41487 invoked by uid 500); 10 Oct 2012 15:20:21 -0000 Delivered-To: apmail-hadoop-mapreduce-user-archive@hadoop.apache.org Received: (qmail 41380 invoked by uid 500); 10 Oct 2012 15:20:20 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 41372 invoked by uid 99); 10 Oct 2012 15:20:20 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Oct 2012 15:20:20 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of shin1.ym@gmail.com designates 209.85.216.48 as permitted sender) Received: from [209.85.216.48] (HELO mail-qa0-f48.google.com) (209.85.216.48) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Oct 2012 15:20:16 +0000 Received: by mail-qa0-f48.google.com with SMTP id c11so649066qad.14 for ; Wed, 10 Oct 2012 08:19:55 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=nH9hqcEsO/Su5oIUMei7naPJsjHO3t9hnIKT+4aQXUs=; b=QSHhq3bmTbxts5bWKKUHPEDmDjvCjzXo3558QbpHq+aqr8cXm4gRPEGCx/uHmoHpWT +YPF5s1JYj1IP7Wyx9lXomJoyAfAaeuTPN+Uzk2whJSFdgds2Srw0/qxZXwdq8W/hBs9 Rga79N9aU2vzax3hP9pGtsTHjzpgt3YJxXyN3wJioFMTCvFTf0ie9Zt/kP6TnB+HNm6p PKNySylvKxLzj/udIJca9faZ/GQYo1KeyMbApHms3FajuIbyVA1umi4Sy/eh9GmU588w JCUQrbxngqUF15Z4DVN8T6J12mkZfsAaFtV437Zm/DqdKYAJOPA7MbW70CVS1dzg/KFm XJFA== MIME-Version: 1.0 Received: by 10.229.106.139 with SMTP id x11mr629708qco.35.1349882395384; Wed, 10 Oct 2012 08:19:55 -0700 (PDT) Received: by 10.49.27.196 with HTTP; Wed, 10 Oct 2012 08:19:55 -0700 (PDT) In-Reply-To: References: Date: Thu, 11 Oct 2012 00:19:55 +0900 Message-ID: Subject: Re: How to change topology From: Shinichi Yamashita To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=002354471320aa5ca004cbb5fc28 X-Virus-Checked: Checked by ClamAV on apache.org --002354471320aa5ca004cbb5fc28 Content-Type: text/plain; charset=ISO-8859-1 Hi Steve, Thank you for sharing information of JIRA with me. Topology easy setting and confirmation will be necessary. I use JIRA for an idea about toplogy setting and confirmation. Regards, Shinichi 2012/10/10 Ted Dunning > > > On Tue, Oct 9, 2012 at 12:17 PM, Steve Loughran wrote: > >> >> >> On 9 October 2012 16:51, Shinichi Yamashita wrote: >> >>> Hi Steve, >>> >>> Thank you for your reply. >>> >>> >>> > no, it's the Namenode and JobTracker that needs to be restarted; >>> > they are the bits that care where the boxes are. >>> >>> I confirmed it in my cluster, and I understood it as follows. >>> First, the resolved node information is recorded in ConcurrentHashMap. >>> Next same node toplogy decision, HashMap data is given priority to over >>> a script. >>> The HashMap data become invalid by restart of NameNode and JobTracker. >>> >> >> That's right. There's been lots of discussion on adding live topology >> resetting. >> > > For what it's worth, there is at least one Hadoop derived distribution > which has live topology resetting already. > > I can follow up offline with anyone who wants more information. > > --002354471320aa5ca004cbb5fc28 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Steve,

Thank you for sharing information of JIRA with me.
Topo= logy easy setting and confirmation will be necessary.
I use JIRA for an = idea about toplogy setting and confirmation.

Regards,
Shinichi
2012/10/10 Ted Dunning <= ;tdunning@maprte= ch.com>


On Tue, Oct 9, 2012 at= 12:17 PM, Steve Loughran <stevel@hortonworks.com> wrot= e:


On 9 October 2012 16:51, Shinichi Yamash= ita <shin1.ym@gmail.com> wrote:
Hi Steve,

Thank you for your reply.


> no, it's th= e Namenode and JobTracker that needs to be restarted;
> they are the= bits that care where the boxes are.

I confirmed it in my clus= ter, and I understood it as follows.
First, the resolved node information is recorded in ConcurrentHashMap.
N= ext same node toplogy decision, HashMap data is given priority to over a sc= ript.
The HashMap data become invalid by restart of NameNode and JobTrac= ker.

That's right. There's been lots of= discussion on adding live topology resetting.=A0
<= div>
For what it's worth, there is at least one Had= oop derived distribution which has live topology resetting already.

I can follow up offline with anyone who wants more info= rmation.


--002354471320aa5ca004cbb5fc28--