hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Davey Yan <davey....@gmail.com>
Subject DataBlockScanner's rate limit
Date Fri, 02 Aug 2013 01:55:25 GMT
I recently got a mini cluster corrupted after my inappropriate process.

This mini cluster's dfs.replication was set to 1.
After irregular restart of OS, I cannot wait to leave safemode, the block
ratio is 0.9862, < 0.999.
In the http://ip:50075/blockScannerReport, I notice there is rate limit to
1MB.
It will verify the blocks for long time.

So I "hadoop dfdsadmin safemode leave", and then I got blocks missing.

My question is: Why should we limit the rate in DataBlockScanner while the
cluster is still starting up or still in safemode?

I read the source code of DataBlockScanner.java, there is no parameter to
change the rate limit.
It seams to be 1MB to 8MB always.


-- 
Davey Yan

Mime
View raw message