Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of paul.m.wilkinson@gmail.com
 designates 209.85.212.179 as permitted sender)
References: <EF2F5708-5B6F-414B-809E-28A9661626EA@oit.gatech.edu>
 <CADokdcOjoxv3VptiQDZbWdpEZCgO0PVC6_O0x-X7RF3sd29_kA@mail.gmail.com>
In-Reply-To: 
 <CADokdcOjoxv3VptiQDZbWdpEZCgO0PVC6_O0x-X7RF3sd29_kA@mail.gmail.com>
Mime-Version: 1.0 (1.0)
Content-Transfer-Encoding: 7bit
Content-Type: multipart/alternative;
	boundary=Apple-Mail-BCB6B0B1-4F69-45CA-8166-66159D3754CF
Message-Id: <85DD9E5D-5C53-44BF-A1EB-13CDB31E42A7@gmail.com>
Cc: "user@hadoop.apache.org" <user@hadoop.apache.org>
From: Paul Wilkinson <paul.m.wilkinson@gmail.com>
Subject: Re: Using NFS mounted volume for Hadoop installation/configuration
Date: Mon, 18 Feb 2013 18:44:19 +0000
To: "user@hadoop.apache.org" <user@hadoop.apache.org>


--Apple-Mail-BCB6B0B1-4F69-45CA-8166-66159D3754CF
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable

That requirement for 100% availability is the issue. If NFS goes down, you l=
ose all sorts of things that are critical. This will work for a dev cluster,=
 but strongly isn't recommended for production.=20

As a first step, consider rsync - that way everything is local, so fewer ext=
ernal dependencies. After that, consider not managing boxes by hand :)

Paul


On 18 Feb 2013, at 18:09, Chris Embree <cembree@gmail.com> wrote:

> I'm doing that currently.  No problems to report so far.  =20
>=20
> The only pitfall I've found is around NFS stability.  If your NAS is 100% s=
olid no problems.  I've seen mtab get messed up and refuse to remount if NFS=
 has any hiccups.=20
>=20
> If you want to really crazy, consider NFS for your datanode root fs.  See t=
he oneSIS project for details.  http://onesis.sourceforge.net
>=20
> Enjoy.
>=20
> On Mon, Feb 18, 2013 at 1:00 PM, Mehmet Belgin <mehmet.belgin@oit.gatech.e=
du> wrote:
>> Hi Everyone,
>>=20
>> Will it be any problem if I put the hadoop executables and configuration o=
n a NFS volume, which is shared by all masters and slaves? This way the conf=
iguration changes will be available for all nodes, without need for synching=
 any files. While this looks almost like a no-brainer, I am wondering if the=
re are any pitfalls I need to be aware of.
>>=20
>> On a related question, is there a best practices (do's and don'ts ) docum=
ent that you can suggest other than the regular documentation by Apache?
>>=20
>> Thanks!
>> -Mehmet
>=20

--Apple-Mail-BCB6B0B1-4F69-45CA-8166-66159D3754CF
Content-Type: text/html;
	charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; charset=3D=
utf-8"></head><body dir=3D"auto"><div><span></span></div><div><meta http-equ=
iv=3D"content-type" content=3D"text/html; charset=3Dutf-8"><div>That require=
ment for 100% availability is the issue. If NFS goes down, you lose all sort=
s of things that are critical.&nbsp;This will work for a dev cluster, but st=
rongly isn't recommended for production.&nbsp;</div><div><br></div><div>As a=
 first step, consider rsync - that way everything is local, so fewer externa=
l dependencies. After that, consider not managing boxes by hand :)</div><div=
><br></div><div>Paul</div><div><br></div><div><br>On 18 Feb 2013, at 18:09, C=
hris Embree &lt;<a href=3D"mailto:cembree@gmail.com">cembree@gmail.com</a>&g=
t; wrote:<br><br></div><blockquote type=3D"cite"><div>I'm doing that current=
ly. &nbsp;No problems to report so far. &nbsp;&nbsp;<div><br></div><div>The o=
nly pitfall I've found is around NFS stability. &nbsp;If your NAS is 100% so=
lid no problems. &nbsp;I've seen mtab get messed up and refuse to remount if=
 NFS has any hiccups.&nbsp;</div>
<div><br></div><div>If you want to really crazy, consider NFS for your datan=
ode root fs. &nbsp;See the oneSIS project for details. &nbsp;<a href=3D"http=
://onesis.sourceforge.net">http://onesis.sourceforge.net</a></div><div><br><=
/div>
<div>Enjoy.<br><br><div class=3D"gmail_quote">On Mon, Feb 18, 2013 at 1:00 P=
M, Mehmet Belgin <span dir=3D"ltr">&lt;<a href=3D"mailto:mehmet.belgin@oit.g=
atech.edu" target=3D"_blank">mehmet.belgin@oit.gatech.edu</a>&gt;</span> wro=
te:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px=
 #ccc solid;padding-left:1ex">Hi Everyone,<br>
<br>
Will it be any problem if I put the hadoop executables and configuration on a=
 NFS volume, which is shared by all masters and slaves? This way the configu=
ration changes will be available for all nodes, without need for synching an=
y files. While this looks almost like a no-brainer, I am wondering if there a=
re any pitfalls I need to be aware of.<br>

<br>
On a related question, is there a best practices (do's and don'ts ) document=
 that you can suggest other than the regular documentation by Apache?<br>
<br>
Thanks!<br>
<span class=3D"HOEnZb"><font color=3D"#888888">-Mehmet</font></span></blockq=
uote></div><br></div>
</div></blockquote></div></body></html>=

--Apple-Mail-BCB6B0B1-4F69-45CA-8166-66159D3754CF--