hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Muhuan Huang <mhhu...@cs.ucla.edu>
Subject Does io.sort.mb count in the records or just the keys?
Date Sun, 09 Nov 2014 19:54:39 GMT
Hello everyone,

I have a question about the io.sort.mb property. The document says that
io.sort.mb is the total amount of buffer memory to use while sorting files.
My question is that does it include both the keys and values of the records
or just keys (and perhaps some pointers to the values)?

More specifically in the case of terasort where each record is 100 bytes
but the key is only 10 bytes, if io.sort.mb is set to 100, does it mean
that it can support a maximum of 1M records or 10M records?

Thanks a lot!

Muhuan

Mime
View raw message