Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of
 prvs=716cbe5f7=afaris@linkedin.com designates 69.28.149.80 as permitted
 sender)
From: Adam Faris <afaris@linkedin.com>
To: "<user@hadoop.apache.org>" <user@hadoop.apache.org>
Subject: Re: services requiring topology conf
Thread-Topic: services requiring topology conf
Thread-Index: AQHN8B1gMzT86guCCk+TfygIOlwMYJhE5JCA
Date: Fri, 11 Jan 2013 17:15:47 +0000
Message-ID: <56D3236F-224C-4933-8CF9-BE8E8DA1C30F@linkedin.com>
References: 
 <CANZDn9vGSFh=A3PuPXb2cuf1=+LKw6+AaM31G6JGMBikhNWEjw@mail.gmail.com>
In-Reply-To: 
 <CANZDn9vGSFh=A3PuPXb2cuf1=+LKw6+AaM31G6JGMBikhNWEjw@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <951E3CA58102474BA3EEB8DD4A38F645@linkedin.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

A patch was submitted for topology documentation, but it doesn't appear to =
have made it to any releases.  This svn link may help starting at line 1294=
.
=20
http://svn.apache.org/viewvc?view=3Drevision&revision=3D1411359 =20

Assuming you are using hadoop 1.x and not yarn, the topology script only ne=
eds to be on the namenode and jobtracker.  As you have noticed it doesn't h=
urt anything if you copy the script everywhere as the tasktracker and datan=
ode process will ignore it.   Try looking at pdsh for controlling compute n=
odes and pushing files, but be careful as if you type a bad command it's go=
ing to get ran everywhere. http://code.google.com/p/pdsh/

-- Adam


On Jan 11, 2013, at 9:01 AM, Bryan Beaudreault <bbeaudreault@hubspot.com> w=
rote:

> The documentation on topology conf (topology.script.file.name) is a littl=
e sparse, and while we have it working in our cluster I am trying to make i=
t a little easier to configure.
>=20
> Currently we upload a python file and conf file to every node in our clus=
ter.  However I have a feeling that it is only needed on the NameNode(s) an=
d perhaps JobTracker.  I checked the code for DataNode and see no reference=
 to this configuration parameter, but I wanted to check with you all before=
 I stop updating the conf on every one of my nodes.
>=20
> Can anyone confirm whether these configuration files only need to be pres=
ent on the NameNode/JobTracker, or do they need to be on every node in a cl=
uster?
>=20
> Thanks