hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Bockelman <bbock...@cse.unl.edu>
Subject Re: Caching in HDFS C API Client
Date Mon, 14 Jun 2010 14:39:52 GMT
Hey Owen, all,

I find this one handy if you have root access:

http://linux-mm.org/Drop_Caches

echo 3 > /proc/sys/vm/drop_caches

Drops the pagecache, dentries, and inodes.  Without this, you can still get caching effects
doing the normal "read and write large files" if the linux pagecache outsmarts you (and I
don't know about you, but it often outsmarts me...).

Brian

On Jun 14, 2010, at 9:35 AM, Owen O'Malley wrote:

> Indeed. On the terasort benchmark, I had to run intermediate jobs that
> were larger than ram on the cluster to ensure that the data was not
> coming from the file cache.
> 
> -- Owen


Mime
View raw message