hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Duxbury <br...@rapleaf.com>
Subject Big HDFS deletes lead to dead datanodes
Date Tue, 24 Feb 2009 18:16:15 GMT
On occasion, I've deleted a few TB of stuff in DFS at once. I've  
noticed that when I do this, datanodes start taking a really long  
time to check in and ultimately get marked dead. Some time later,  
they'll get done deleting stuff and come back and get unmarked.

I'm wondering, why do deletions get more priority than checking in?  
Ideally we would never see "dead" datanodes from doing deletes.

-Bryan

Mime
View raw message