hive-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ranjith <ranjith.raghuna...@gmail.com>
Subject Hive job tuning
Date Thu, 31 May 2012 02:50:34 GMT
I have been looking at a job that was performing badly. Noticed there were several splits occurring
due to the buffer record limit being reached. I get that the io.sort.mb provides the data
and record buffer for the mapper task. Given that the mapper jvm starts up with 500mb and
the buffer is 300, where does the rest of the 200 mb get assigned to? Is sitting there idle?
If I increase the io.sort mb to the size of the jvm what effects does that have?

Thanks,
Ranjith
Mime
View raw message