hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mag Gam <magaw...@gmail.com>
Subject Re: testing if replication working
Date Mon, 01 Mar 2010 12:38:27 GMT
Thanks for your responses.





On Mon, Mar 1, 2010 at 2:48 AM, Terrence Martin
<tmartin@physics.ucsd.edu> wrote:
> Mag Gam wrote:
>>
>> I just setup my first hadoop cluster with 5 nodes. What is the best
>> way to check if replication is really working? I assume the best way
>> is to power down 2 nodes and see if I can still reach my data?
>>
>> Or are there any others ways?
>>
>> TIA
>>
>
> Well if you run
>
> hadoop fsck /
>
> it will give you a report of the number of replicas, if a file has less than
> your configured replicas it will tell you that you are under replicated.
>
> As for powering off 2 nodes, well that will likely result in a few missing
> blocks unless you have more than 2 replicas of each block. Now if you turn
> off one data node after a while Hadoop will replicate the blocks to the
> remaining 4 nodes, when you turn the 5th back on you will have over
> replicated blocks.
>
> Terrence
>
>

Mime
View raw message