Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of cembree@gmail.com designates
 209.85.128.50 as permitted sender)
MIME-Version: 1.0
Reply-To: chris@embree.us
In-Reply-To: <85DD9E5D-5C53-44BF-A1EB-13CDB31E42A7@gmail.com>
References: <EF2F5708-5B6F-414B-809E-28A9661626EA@oit.gatech.edu>
	<CADokdcOjoxv3VptiQDZbWdpEZCgO0PVC6_O0x-X7RF3sd29_kA@mail.gmail.com>
	<85DD9E5D-5C53-44BF-A1EB-13CDB31E42A7@gmail.com>
Date: Mon, 18 Feb 2013 14:31:17 -0500
Message-ID: 
 <CADokdcOqdN=GwtA3RCaOZH61RU2ADy_qRVBRab7052r7MqB4mg@mail.gmail.com>
Subject: Re: Using NFS mounted volume for Hadoop installation/configuration
From: Chris Embree <cembree@gmail.com>
To: user@hadoop.apache.org
Content-Type: multipart/alternative; boundary=20cf300faf81e82d8504d604c4bd

--20cf300faf81e82d8504d604c4bd
Content-Type: text/plain; charset=ISO-8859-1

Just for clarification, we only use NFS for binaries and config files.
 HDFS and MarpRed write to local disk.  We just don't install an OS there.
:)

On Mon, Feb 18, 2013 at 1:44 PM, Paul Wilkinson
<paul.m.wilkinson@gmail.com>wrote:

> That requirement for 100% availability is the issue. If NFS goes down, you
> lose all sorts of things that are critical. This will work for a dev
> cluster, but strongly isn't recommended for production.
>
> As a first step, consider rsync - that way everything is local, so fewer
> external dependencies. After that, consider not managing boxes by hand :)
>
> Paul
>
>
> On 18 Feb 2013, at 18:09, Chris Embree <cembree@gmail.com> wrote:
>
> I'm doing that currently.  No problems to report so far.
>
> The only pitfall I've found is around NFS stability.  If your NAS is 100%
> solid no problems.  I've seen mtab get messed up and refuse to remount if
> NFS has any hiccups.
>
> If you want to really crazy, consider NFS for your datanode root fs.  See
> the oneSIS project for details.  http://onesis.sourceforge.net
>
> Enjoy.
>
> On Mon, Feb 18, 2013 at 1:00 PM, Mehmet Belgin <
> mehmet.belgin@oit.gatech.edu> wrote:
>
>> Hi Everyone,
>>
>> Will it be any problem if I put the hadoop executables and configuration
>> on a NFS volume, which is shared by all masters and slaves? This way the
>> configuration changes will be available for all nodes, without need for
>> synching any files. While this looks almost like a no-brainer, I am
>> wondering if there are any pitfalls I need to be aware of.
>>
>> On a related question, is there a best practices (do's and don'ts )
>> document that you can suggest other than the regular documentation by
>> Apache?
>>
>> Thanks!
>> -Mehmet
>
>
>

--20cf300faf81e82d8504d604c4bd
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Just for clarification, we only use NFS for binaries and config files. =A0H=
DFS and MarpRed write to local disk. =A0We just don&#39;t install an OS the=
re. :)<br><br><div class=3D"gmail_quote">On Mon, Feb 18, 2013 at 1:44 PM, P=
aul Wilkinson <span dir=3D"ltr">&lt;<a href=3D"mailto:paul.m.wilkinson@gmai=
l.com" target=3D"_blank">paul.m.wilkinson@gmail.com</a>&gt;</span> wrote:<b=
r>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex"><div dir=3D"auto"><div><span></span></div><d=
iv><div>That requirement for 100% availability is the issue. If NFS goes do=
wn, you lose all sorts of things that are critical.=A0This will work for a =
dev cluster, but strongly isn&#39;t recommended for production.=A0</div>
<div><br></div><div>As a first step, consider rsync - that way everything i=
s local, so fewer external dependencies. After that, consider not managing =
boxes by hand :)</div><span class=3D"HOEnZb"><font color=3D"#888888"><div><=
br>
</div><div>Paul</div></font></span><div><div class=3D"h5"><div><br></div><d=
iv><br>On 18 Feb 2013, at 18:09, Chris Embree &lt;<a href=3D"mailto:cembree=
@gmail.com" target=3D"_blank">cembree@gmail.com</a>&gt; wrote:<br><br></div=
><blockquote type=3D"cite">
<div>I&#39;m doing that currently. =A0No problems to report so far. =A0=A0<=
div><br></div><div>The only pitfall I&#39;ve found is around NFS stability.=
 =A0If your NAS is 100% solid no problems. =A0I&#39;ve seen mtab get messed=
 up and refuse to remount if NFS has any hiccups.=A0</div>

<div><br></div><div>If you want to really crazy, consider NFS for your data=
node root fs. =A0See the oneSIS project for details. =A0<a href=3D"http://o=
nesis.sourceforge.net" target=3D"_blank">http://onesis.sourceforge.net</a><=
/div>
<div><br></div>
<div>Enjoy.<br><br><div class=3D"gmail_quote">On Mon, Feb 18, 2013 at 1:00 =
PM, Mehmet Belgin <span dir=3D"ltr">&lt;<a href=3D"mailto:mehmet.belgin@oit=
.gatech.edu" target=3D"_blank">mehmet.belgin@oit.gatech.edu</a>&gt;</span> =
wrote:<br>

<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">Hi Everyone,<br>
<br>
Will it be any problem if I put the hadoop executables and configuration on=
 a NFS volume, which is shared by all masters and slaves? This way the conf=
iguration changes will be available for all nodes, without need for synchin=
g any files. While this looks almost like a no-brainer, I am wondering if t=
here are any pitfalls I need to be aware of.<br>


<br>
On a related question, is there a best practices (do&#39;s and don&#39;ts )=
 document that you can suggest other than the regular documentation by Apac=
he?<br>
<br>
Thanks!<br>
<span><font color=3D"#888888">-Mehmet</font></span></blockquote></div><br><=
/div>
</div></blockquote></div></div></div></div></blockquote></div><br>

--20cf300faf81e82d8504d604c4bd--