hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Doug Cutting <cutt...@apache.org>
Subject Re: InputFiles, Splits, Maps, Tasks Questions 1.3 Base
Date Thu, 18 Oct 2007 22:21:47 GMT
Lance Amundsen wrote:
> There's lots of references on decreasing DFS block size to increase maps to
> record ratios.  What is the easiest way to do this?  Is it possible with
> the standard SequenceFile class?

You could specify the block size in the Configuration parameter to 
SequenceFile#createWriter() using the dfs.block.size parameter.  But if 
you simply want to create sub-block-size splits, then increasing the 
number of map tasks should do that.


View raw message