Mailing-List: contact user-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hadoop.apache.org
Received-SPF: pass (athena.apache.org: domain of bbeaudreault@hubspot.com
 designates 74.125.149.201 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAPQV63VOa66LK1AUkTUHeUHJ-ToRHMHhjx1hVqvNNjJqrdtrtA@mail.gmail.com>
References: 
 <CAPQV63VOa66LK1AUkTUHeUHJ-ToRHMHhjx1hVqvNNjJqrdtrtA@mail.gmail.com>
From: Bryan Beaudreault <bbeaudreault@hubspot.com>
Date: Tue, 18 Dec 2012 15:36:25 -0500
Message-ID: 
 <CANZDn9vFTTNevY-__3CRVcyXkzriKR12TPN8nsYdFZuQfa0oQg@mail.gmail.com>
Subject: Re: Misconfiguration of hdfs-site.xml
To: "hbase-user@hadoop.apache.org" <user@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=047d7b2e0cabd1946104d12674a6

--047d7b2e0cabd1946104d12674a6
Content-Type: text/plain; charset=ISO-8859-1

Updating the config and restarting will result in data loss.  You're going
to want to migrate the data from your tmp directory to the appropriate
place.

The NameNode will use the dfs.name.dir property, and DataNodes will use the
dfs.data.dir.  Assuming you aren't running a DN and NN on the same server
(don't do that), then they shouldn't collide/overlap.  Even still its
probably best to use different directories.  For instance mine is like this:

dfs.name.dir = /mnt/hdfs/name,/mnt2/hdfs/name
dfs.data.dir
= /mnt/hdfs/data,/mnt2/hdfs/data,/mnt3/hdfs/data,/mnt4/hdfs/data

Hope this helps.


On Tue, Dec 18, 2012 at 3:28 PM, Jean-Marc Spaggiari <
jean-marc@spaggiari.org> wrote:

> Hi,
>
> For months now I'm using my hadoop cluster with absolutly nothing
> related to the drive directory on my hdfs-site.xml file.
>
> It seems that it's using the hadoop.tmp.dir directory to store data.
>
> My hadoop.tmp.dir is pointing to
> /home/hadoop/haddop_drive/${user.name} and on my
> /home/hadoop/haddop_drive directory I can see hadoop and hbase.
>
> Now, I want to configure that properly without loosing the data I have.
>
> I'm using 1.0.3.
>
> Based on the documentation
> http://hadoop.apache.org/docs/r1.0.3/cluster_setup.html#Site+Configuration
> it seems I need to setup dfs.data.dir to point to
> /home/hadoop/haddop_drive. If I add this entry and restart my
> datanode, will it "simply" continue to point to the same place and
> find the data he needs?
>
> Also, what about dfs.name.dir ? I guess I can simply point it to the
> same place, right?
>
> On  /home/hadoop/haddop_drive/hadoop/dfs I have data and name
> directories. So I'm not sure if I should point the properties above to
>  /home/hadoop/haddop_drive or to  /home/hadoop/haddop_drive/dfs.
>
> Thanks,
>
> JM
>

--047d7b2e0cabd1946104d12674a6
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

Updating the config and restarting will result in data loss. =A0You&#39;re =
going to want to migrate the data from your tmp directory to the appropriat=
e place.<div><br></div><div>The NameNode will use the dfs.name.dir property=
, and DataNodes will use the dfs.data.dir. =A0Assuming you aren&#39;t runni=
ng a DN and NN on the same server (don&#39;t do that), then they shouldn=
9;t collide/overlap. =A0Even still its probably best to use different direc=
tories. =A0For instance mine is like this:</div>

<div><br></div><div>dfs.name.dir =3D /mnt/hdfs/name,/mnt2/hdfs/name</div><d=
iv>dfs.data.dir =3D=A0/mnt/hdfs/data,/mnt2/hdfs/data,/mnt3/hdfs/data,/mnt4/=
hdfs/data</div><div><br></div><div>Hope this helps.</div><div class=3D"gmai=
l_extra">

<br><br><div class=3D"gmail_quote">On Tue, Dec 18, 2012 at 3:28 PM, Jean-Ma=
rc Spaggiari <span dir=3D"ltr">&lt;<a href=3D"mailto:jean-marc@spaggiari.or=
g" target=3D"_blank">jean-marc@spaggiari.org</a>&gt;</span> wrote:<br><bloc=
kquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #cc=
c solid;padding-left:1ex">

Hi,<br>
<br>
For months now I&#39;m using my hadoop cluster with absolutly nothing<br>
related to the drive directory on my hdfs-site.xml file.<br>
<br>
It seems that it&#39;s using the hadoop.tmp.dir directory to store data.<br=
>
<br>
My hadoop.tmp.dir is pointing to<br>
/home/hadoop/haddop_drive/${<a href=3D"http://user.name" target=3D"_blank">=
user.name</a>} and on my<br>
/home/hadoop/haddop_drive directory I can see hadoop and hbase.<br>
<br>
Now, I want to configure that properly without loosing the data I have.<br>
<br>
I&#39;m using 1.0.3.<br>
<br>
Based on the documentation<br>
<a href=3D"http://hadoop.apache.org/docs/r1.0.3/cluster_setup.html#Site+Con=
figuration" target=3D"_blank">http://hadoop.apache.org/docs/r1.0.3/cluster_=
setup.html#Site+Configuration</a><br>
it seems I need to setup dfs.data.dir to point to<br>
/home/hadoop/haddop_drive. If I add this entry and restart my<br>
datanode, will it &quot;simply&quot; continue to point to the same place an=
d<br>
find the data he needs?<br>
<br>
Also, what about dfs.name.dir ? I guess I can simply point it to the<br>
same place, right?<br>
<br>
On =A0/home/hadoop/haddop_drive/hadoop/dfs I have data and name<br>
directories. So I&#39;m not sure if I should point the properties above to<=
br>
=A0/home/hadoop/haddop_drive or to =A0/home/hadoop/haddop_drive/dfs.<br>
<br>
Thanks,<br>
<br>
JM<br>
</blockquote></div><br></div>

--047d7b2e0cabd1946104d12674a6--