hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From highpointe <highpoint...@gmail.com>
Subject DFS maxing out on single datadir
Date Thu, 19 May 2011 01:37:35 GMT
System:  HDFS dirs across cluster as dataone, datatwo, datathree

I recently had an issue where I lost a slave that resulted in a large amount of under replicated

The replication was quite slow on the uptake so I thought running a hadoop balancer would

This seemed to exacerbate the situation so I killed the balancer. 

Hadoop then proceeded to write all new data to dataone across each slave. It would wait until
the dataone dir was at 100% then move to the next slave in sequence. datatwo and datathree
were completely ignored.

DFS showed <10% free and was quickly diving.

I ended up restarting the entire cluster (DFS and MapRed) and things started acting normal
again (writing to all three replicants).

Has anyone experienced this or have any idea why it would happen?

Thanks for the help. 

Sent from my iPhone
View raw message