Return-Path: X-Original-To: apmail-hadoop-user-archive@minotaur.apache.org Delivered-To: apmail-hadoop-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 278EBDAFF for ; Tue, 9 Oct 2012 15:51:38 +0000 (UTC) Received: (qmail 23879 invoked by uid 500); 9 Oct 2012 15:51:33 -0000 Delivered-To: apmail-hadoop-user-archive@hadoop.apache.org Received: (qmail 23781 invoked by uid 500); 9 Oct 2012 15:51:33 -0000 Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hadoop.apache.org Delivered-To: mailing list user@hadoop.apache.org Received: (qmail 23774 invoked by uid 99); 9 Oct 2012 15:51:33 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Oct 2012 15:51:33 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of shin1.ym@gmail.com designates 209.85.216.41 as permitted sender) Received: from [209.85.216.41] (HELO mail-qa0-f41.google.com) (209.85.216.41) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 09 Oct 2012 15:51:26 +0000 Received: by mail-qa0-f41.google.com with SMTP id p27so3911651qat.14 for ; Tue, 09 Oct 2012 08:51:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=Zep1ZmpYMAE5oT6G+gxONH5BDIP0A+0SfXfX+eTq83s=; b=GLrha4HUUPW6BQltxVYi6X3Q07wk1fk/C/LyqMLuS9yHWAhzoy4AR3FMVjIL7Eeu+g JPlqrMo+XgMHt3TZJKYOk5Rzj0Z0zvoG+W11EKfwXXlh1Mg9w7pDYfuP/GzR4kLUeuGr 2FZ6Oq4uuAwHz3736j7KXqco+yoMK+0pXjFCGkZ6NRyGmBMkjcKmSphs8QNd1cbYzkeq KealPgOCVVrKVwQDk1jCb7fzoxJsEDR2RhgWwa9fsjqXKM1mv3CBpAJNid2rPRhvGwzo GXbeO/XR8ahEuf1Rxad4U/o59l8buVyt995QISCPQG150PYTvDWu3CqaRuEOfP3lo8i2 7jqw== MIME-Version: 1.0 Received: by 10.224.217.136 with SMTP id hm8mr18754203qab.81.1349797865081; Tue, 09 Oct 2012 08:51:05 -0700 (PDT) Received: by 10.49.27.196 with HTTP; Tue, 9 Oct 2012 08:51:04 -0700 (PDT) In-Reply-To: References: Date: Wed, 10 Oct 2012 00:51:04 +0900 Message-ID: Subject: Re: How to change topology From: Shinichi Yamashita To: user@hadoop.apache.org Content-Type: multipart/alternative; boundary=20cf300fb1dd444bdf04cba24e18 --20cf300fb1dd444bdf04cba24e18 Content-Type: text/plain; charset=ISO-8859-1 Hi Steve, Thank you for your reply. > no, it's the Namenode and JobTracker that needs to be restarted; > they are the bits that care where the boxes are. I confirmed it in my cluster, and I understood it as follows. First, the resolved node information is recorded in ConcurrentHashMap. Next same node toplogy decision, HashMap data is given priority to over a script. The HashMap data become invalid by restart of NameNode and JobTracker. Regards, Shinichi 2012/10/9 Steve Loughran > > > On 8 October 2012 14:23, Shinichi Yamashita wrote: > >> Hi, >> >> I know that DataNode and TaskTracker must restart to change topology. >> > > no, it's the Namenode and JobTracker that needs to be restarted; they are > the bits that care where the boxes are. > > >> Is there the method to execute the topology change without restart of >> DataNode and TaskTracker? >> In other words, can I change the topology by a command? >> > > No -the map gets cached, and when the services start up they do quick > check to see if the topology is flat or spread across switches -and use > that flag to make various placement decisions. > > steve > > > --20cf300fb1dd444bdf04cba24e18 Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi Steve,

Thank you for your reply.

> no, it's the Nam= enode and JobTracker that needs to be restarted;
> they are the bits= that care where the boxes are.

I confirmed it in my cluster, and I = understood it as follows.
First, the resolved node information is recorded in ConcurrentHashMap.
N= ext same node toplogy decision, HashMap data is given priority to over a sc= ript.
The HashMap data become invalid by restart of NameNode and JobTrac= ker.

Regards,
Shinichi=A0

2012/10/9 St= eve Loughran <stevel@hortonworks.com>


On 8 October 2012 14:2= 3, Shinichi Yamashita <shin1.ym@gmail.com> wrote:
Hi,

I know that DataNode and TaskTracker must restart to change topo= logy.

no, it's the Namenode a= nd JobTracker that needs to be restarted; they are the bits that care where= the boxes are.
=A0
Is there the method to execute= the topology change without restart of DataNode and TaskTracker?
In oth= er words, can I change the topology by a command?

No -the map gets cached, and when th= e services start up they do quick check to see if the topology is flat or s= pread across switches -and use that flag to make various placement decision= s.

steve=A0
--20cf300fb1dd444bdf04cba24e18--