hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patai Sangbutsarakum <Patai.Sangbutsara...@turn.com>
Subject Re: adding space on existing datanode ?
Date Fri, 22 Feb 2013 18:56:09 GMT
Just want to add up from JM.

If you already have balancer run in cluster every day, that will help the new drive(s) get


From: Jean-Marc Spaggiari <jean-marc@spaggiari.org<mailto:jean-marc@spaggiari.org>>
Reply-To: <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Date: Fri, 22 Feb 2013 13:14:14 -0500
To: <user@hadoop.apache.org<mailto:user@hadoop.apache.org>>
Subject: Re: adding space on existing datanode ?

 add disk space to you datanode you simply need to add another drive, then add it to the dfs.data.dir
or dfs.datanode.data.dir entry. After a datanode restart, hadoop will start to use it.

It will not balance the existing data between the directories. It will continue to add to
the 2. If one goes full, it will only continue with the other one. If required, you can balance
the data manually. Or depending on your use case and the options you have, you can stop the
datanode, delete the content of the 2 data directories and restart it. It will stat to receive
data to duplicate and will share it evenly between the 2 directories. This last solution is
not recommended. But for a test environment it might be easier.

View raw message