hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ellis H. Wilson III" <el...@cse.psu.edu>
Subject fs.local.block.size vs file.blocksize
Date Thu, 09 Aug 2012 13:10:03 GMT
Hi all!

Can someone please briefly explain the difference?  I do not see 
deprecated warnings for fs.local.block.size when I run with them set and 
I see two copies of RawLocalFileSystem.java (the other is 
local/RawLocalFs.java).

The things I really need to get answers to are:
1. Is the default boosted to 64MB from Hadoop 1.0 to Hadoop 2.0?  I 
believe it is, but want validation on that.
2. Which one controls shuffle block-size?
3. If I have a single machine non-distributed instance, and point it at 
file://, do both of these control the persistent data's block size or 
just one of them or what?
4. Is there any way to run with say a 512MB blocksize for the persistent 
data and the default 64MB blocksize for the shuffled data?

Thanks!

ellis

Mime
View raw message