hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: Supersede a data node help: how to move all files out of a Hadoop data node?
Date Fri, 20 Aug 2010 20:39:19 GMT
On Fri, Aug 20, 2010 at 4:31 PM, jiang licht <licht_jiang@yahoo.com> wrote:
> Requirement: I want to get rid of a data node machine. But it has useful data that is
still in use. So, I want to move all its files/blocks to other live data nodes in the same
> Question: I understand that if a data node is down for a certain amount of time, it will
be marked as "dead" and Hadoop will automatically generate a new replication on another live
data node for each block on the dead node. So, sooner or later, all files/blocks on the dead
data node will be replicated (or "moved") to other data node machines.  My question is:
> Is it possible that this process can be explicitly controlled such that I know when all
the missing blocks on the dead node are replicated to other live nodes?
> What is the recommended way to do this?
> How to check there is no missing blocks (or under replication)?
> Thanks,
> Michael

If you run a 'hadoop fsck /' one of the things reported is Under
replicated blocks. When under replicated blocks =0 everything is
moved. (assuming there are not other problems)

View raw message