incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: Help on MMap of SSTables
Date Thu, 06 Dec 2012 02:49:29 GMT
Background http://en.wikipedia.org/wiki/Memory-mapped_file

> Is it going to load only relevant pages per SSTable on read or is it going to load an
entire SSTable on first access?
It will load what is requested, and maybe some additional data taking into account the amount
of memory available for caches. 

> Say suppose compaction kicks in. Will it then evict hot MMapped pages for read and substitute
it with a lot of pages involving full SSTables?

Some file access in cassandra, such as compaction, hints to the OS that the reads should not
be cached. Technically is uses posix_fadvise if you want to look it up.

Cheers


-----------------
Aaron Morton
Freelance Cassandra Developer
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 5/12/2012, at 11:04 PM, Ravikumar Govindarajan <ravikumar.govindarajan@gmail.com>
wrote:

> Thanks Aaron,
> 
> I am not quite clear on how MMap loads SSTables other than the fact that it kicks in
only during a first-time access
> 
> Is it going to load only relevant pages per SSTable on read or is it going to load an
entire SSTable on first access?
> 
> Say suppose compaction kicks in. Will it then evict hot MMapped pages for read and substitute
it with a lot of pages involving full SSTables?
> 
> --
> Ravi
> 
> On Wed, Dec 5, 2012 at 1:22 AM, aaron morton <aaron@thelastpickle.com> wrote:
>> Will MMapping data files be detrimental for reads, in this case?
> No. 
> 
>> In general, when should we opt for MMap data files and what are the factors that
need special attention when enabling the same?
> mmapping is the default, so I would say use it until you have a reason not to. 
> 
> mmapping will map the entire file, but pages of data are read into memory on demand and
purged when space is needed. 
> 
> Cheers
> 
> -----------------
> Aaron Morton
> Freelance Cassandra Developer
> New Zealand
> 
> @aaronmorton
> http://www.thelastpickle.com
> 
> On 4/12/2012, at 11:59 PM, Ravikumar Govindarajan <ravikumar.govindarajan@gmail.com>
wrote:
> 
>> Our current SSTable sizes are far greater than RAM. {150 Gigs of data, 32GB RAM}.
Currently we run with mlockall and mmap_index_only options and don't experience swapping at
all.
>> 
>> We use wide rows and size-tiered-compaction, so a given key will definitely be spread
across multiple sstables. Will MMapping data files be detrimental for reads, in this case?
>> 
>> In general, when should we opt for MMap data files and what are the factors that
need special attention when enabling the same?
>> 
>> --
>> Ravi
> 
> 


Mime
View raw message