Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
From: Chris MacKenzie <studio@chrismackenziephotography.co.uk>
Content-Type: multipart/alternative;
	boundary=Apple-Mail-F3479B58-577E-4A47-9089-BD1602234231
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0 (1.0)
Subject: Re: Hadoop YARM Cluster Setup Questions
Message-Id: 
 <46A3DA43-2BCF-4D8A-BEC0-380E9A2B2BED@chrismackenziephotography.co.uk>
Date: Sat, 23 Aug 2014 17:44:32 +0100
References: <53f8ad1d.888d320a.7b24.7f67@mx.google.com>
In-Reply-To: <53f8ad1d.888d320a.7b24.7f67@mx.google.com>
To: "user@hadoop.apache.org" <user@hadoop.apache.org>


--Apple-Mail-F3479B58-577E-4A47-9089-BD1602234231
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable

Hi,

The requirement is simply to have the slaves and masters files on the resour=
ce manager it's used by the shell script that starts the demons :-)

Sent from my iPhone

> On 23 Aug 2014, at 16:02, "S.L" <simpleliving016@gmail.com> wrote:
>=20
> Ok, Ill copy the slaves file to the other slave nodes as well.
>=20
> What about the masters file though?
>=20
> Sent from my HTC
>=20
> ----- Reply message -----
> From: "rab ra" <rabmdu@gmail.com>
> To: "user@hadoop.apache.org" <user@hadoop.apache.org>
> Subject: Hadoop YARM Cluster Setup Questions
> Date: Sat, Aug 23, 2014 5:03 AM
>=20
> Hi,
>=20
> 1. Typically,we used to copy the slaves file all the participating nodes
> though I do not have concrete theory to back up this. Atleast, this is wha=
t
> I was doing in hadoop 1.2 and I am doing the same in hadoop 2x
>=20
> 2. I think, you should investigate the yarn GUI and see how many maps it
> has spanned. There is a high possibility that both the maps are running in=

> the same node in parallel. Since there are two splits, there would be two
> map processes, and one node is capable of handling more than one map.
>=20
> 3. There could be no replica of input file stored and it is small, and
> hence stored in a single block in one node itself.
>=20
> These could be few hints which might help you
>=20
> regards
> rab
>=20
>=20
>=20
> On Sat, Aug 23, 2014 at 12:26 PM, S.L <simpleliving016@gmail.com> wrote:
>=20
> > Hi Folks,
> >
> > I was not able to find  a clear answer to this , I know that on the mast=
er
> > node we need to have a slaves file listing all the slaves , but do we ne=
ed
> > to have the slave nodes have a master file listing the single name node(=
 I
> > am not using a secondary name node). I only have the slaves file on the
> > master node.
> >
> > I was not able to find a clear answer to this ,the reason I ask this is
> > because when I submit a hadoop job , even though the input is being spli=
t
> > into 2 parts , only one data node is assigned applications , the other t=
wo
> > ( I have three) are no tbeing assigned any applications.
> >
> > Thanks in advance!
> >

--Apple-Mail-F3479B58-577E-4A47-9089-BD1602234231
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: 7bit

<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto"><div>Hi,</div><div><br></div><div>The requirement is simply to have the slaves and masters files on the resource manager it's used by the shell script that starts the demons :-)<br><br>Sent from my iPhone</div><div><br>On 23 Aug 2014, at 16:02, "S.L" &lt;<a href="mailto:simpleliving016@gmail.com">simpleliving016@gmail.com</a>&gt; wrote:<br><br></div><blockquote type="cite"><div>
<div style="font-size: 12pt; font-family: Calibri,sans-serif;"><div>Ok, Ill copy the slaves file to the other slave nodes as well.</div><div><br></div><div>What about the masters file though?</div><div><br></div><div>Sent from my HTC</div><br><div id="htc_header">----- Reply message -----<br>From: "rab ra" &lt;<a href="mailto:rabmdu@gmail.com">rabmdu@gmail.com</a>&gt;<br>To: "<a href="mailto:user@hadoop.apache.org">user@hadoop.apache.org</a>" &lt;<a href="mailto:user@hadoop.apache.org">user@hadoop.apache.org</a>&gt;<br>Subject: Hadoop YARM Cluster Setup Questions<br>Date: Sat, Aug 23, 2014 5:03 AM</div></div><br><pre style="word-wrap: break-word; white-space: pre-wrap;">Hi,

1. Typically,we used to copy the slaves file all the participating nodes
though I do not have concrete theory to back up this. Atleast, this is what
I was doing in hadoop 1.2 and I am doing the same in hadoop 2x

2. I think, you should investigate the yarn GUI and see how many maps it
has spanned. There is a high possibility that both the maps are running in
the same node in parallel. Since there are two splits, there would be two
map processes, and one node is capable of handling more than one map.

3. There could be no replica of input file stored and it is small, and
hence stored in a single block in one node itself.

These could be few hints which might help you

regards
rab


On Sat, Aug 23, 2014 at 12:26 PM, S.L &lt;<a href="mailto:simpleliving016@gmail.com">simpleliving016@gmail.com</a>&gt; wrote:

&gt; Hi Folks,
&gt;
&gt; I was not able to find  a clear answer to this , I know that on the master
&gt; node we need to have a slaves file listing all the slaves , but do we need
&gt; to have the slave nodes have a master file listing the single name node( I
&gt; am not using a secondary name node). I only have the slaves file on the
&gt; master node.
&gt;
&gt; I was not able to find a clear answer to this ,the reason I ask this is
&gt; because when I submit a hadoop job , even though the input is being split
&gt; into 2 parts , only one data node is assigned applications , the other two
&gt; ( I have three) are no tbeing assigned any applications.
&gt;
&gt; Thanks in advance!
&gt;
</pre></div></blockquote></body></html>
--Apple-Mail-F3479B58-577E-4A47-9089-BD1602234231--