hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From brice lecomte <bleco...@astek.fr>
Subject Re: adding space on existing datanode ?
Date Mon, 25 Feb 2013 08:50:29 GMT
Thanks for your reply, I'm running 1.1.1, hence dfs.data.dir looks to be
the right property to add, but doing so, it would add another complete
datanode (incrementing dfs.replication by 1) whereas here, I'd like just
to "extend" an existing one. Am I wrong ?

Le 22/02/2013 19:56, Patai Sangbutsarakum a écrit :
> Just want to add up from JM.
> If you already have balancer run in cluster every day, that will help
> the new drive(s) get balanced.
> P
> From: Jean-Marc Spaggiari <jean-marc@spaggiari.org
> <mailto:jean-marc@spaggiari.org>>
> Reply-To: <user@hadoop.apache.org <mailto:user@hadoop.apache.org>>
> Date: Fri, 22 Feb 2013 13:14:14 -0500
> To: <user@hadoop.apache.org <mailto:user@hadoop.apache.org>>
> Subject: Re: adding space on existing datanode ?
>  add disk space to you datanode you simply need to add another drive,
> then add it to the dfs.data.dir or dfs.datanode.data.dir entry. After
> a datanode restart, hadoop will start to use it.
> It will not balance the existing data between the directories. It will
> continue to add to the 2. If one goes full, it will only continue with
> the other one. If required, you can balance the data manually. Or
> depending on your use case and the options you have, you can stop the
> datanode, delete the content of the 2 data directories and restart it.
> It will stat to receive data to duplicate and will share it evenly
> between the 2 directories. This last solution is not recommended. But
> for a test environment it might be easier. 

View raw message