hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Olimpiati <markq2...@gmail.com>
Subject Hadoop I/O buffer size
Date Fri, 07 Dec 2012 02:35:59 GMT
Hi guys,

   Need help figuring out Hadoop I/O buffer size. This link:
http://developer.yahoo.com/blogs/hadoop/posts/2009/08/the_anatomy_of_hadoop_io_pipel/,
implies that
*io.file.buffersize* is 4K or 64K which makes me wonder. Is that the buffer
which gets filled with records from disk when I have the following in code:

*While( reader.next(Key,Value)) ;  ? *

If yes, when I change io.file.buffersize = 64M will that fetch 64MB of
records into memory when I do "reader.next" ?

if no, then how many records are fetched when I do one
"reader.next(key,value)" ?

Thank you,
Mark

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message