hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: 1.2.1 ouput, what is bufvoid
Date Mon, 26 Aug 2013 22:14:02 GMT
You need to raise io.sort.mb to increase the sort buffer capacity.

On Tue, Aug 27, 2013 at 12:56 AM, kaveh minooie <kaveh@plutoz.com> wrote:
> I am getting these in job output in the 1.2.1
>
> Spilling map output: record full = true
> 2013-08-26 11:40:33,413 INFO org.apache.hadoop.mapred.MapTask: bufstart =
> 111149676; bufend = 122264640; bufvoid = 318767104
> 2013-08-26 11:40:33,413 INFO org.apache.hadoop.mapred.MapTask: kvstart =
> 1048561; kvend = 838844; length = 1048576
> 2013-08-26 11:40:35,879 INFO org.apache.hadoop.mapred.MapTask: Finished
> spill 10
>
>
> my question is what is bufvoid? and how can I increase the amount of the
> length?
>
> thanks,
> --
> Kaveh Minooie



-- 
Harsh J

Mime
View raw message