Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 94910 invoked from network); 23 Feb 2011 09:21:19 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 23 Feb 2011 09:21:19 -0000 Received: (qmail 64827 invoked by uid 500); 23 Feb 2011 09:21:17 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 64681 invoked by uid 500); 23 Feb 2011 09:21:14 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 64673 invoked by uid 99); 23 Feb 2011 09:21:13 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Feb 2011 09:21:13 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of mdennis@riptano.com designates 209.85.213.172 as permitted sender) Received: from [209.85.213.172] (HELO mail-yx0-f172.google.com) (209.85.213.172) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 23 Feb 2011 09:21:04 +0000 Received: by yxk30 with SMTP id 30so1020518yxk.31 for ; Wed, 23 Feb 2011 01:20:43 -0800 (PST) MIME-Version: 1.0 Received: by 10.236.111.5 with SMTP id v5mr6861269yhg.83.1298452842909; Wed, 23 Feb 2011 01:20:42 -0800 (PST) Sender: mdennis@riptano.com Received: by 10.236.105.161 with HTTP; Wed, 23 Feb 2011 01:20:42 -0800 (PST) X-Originating-IP: [184.198.47.128] In-Reply-To: References: Date: Wed, 23 Feb 2011 03:20:42 -0600 X-Google-Sender-Auth: cCt6CsiB-t7DruHQW6bQnrCM5vQ Message-ID: Subject: Re: Reads and memory usage clarification From: Matthew Dennis To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=0023547c8bdf759076049cef9c4d X-Virus-Checked: Checked by ClamAV on apache.org --0023547c8bdf759076049cef9c4d Content-Type: text/plain; charset=ISO-8859-1 Data is in Memtables from writes before they get flushed (based on first threshold of ops/size/time exceeded; all are configurable) to SSTables on disk. There is a keycache and a rowcache. The keycache caches offsets into SSTables for the rows. the rowcache caches the entire row. There is also the OS page cache which is heavily used. When a read happens, the keycache is updated with the information for the SSTables the row was eventually found in. If there are too many entries now in the keycache, some are ejected. Overall the keycache uses very little memory per entry and can cut your disk IO in half so it's a pretty big win. If you read an entire row it goes in the row cache. Like the keycache, this may result in older entries being ejected from the cache. If you insert lots of really large rows in the rowcache you can OOM your JVM. The rowcache is kept up to date with the memtables as writes come in. When a read comes in, C* will collect the data from the SSTables and Memtables and merge them together but data only goes into Memtables from writes. On Tue, Feb 22, 2011 at 3:32 AM, Viktor Jevdokimov wrote: > Hello, > > Write path is perfectly documented in architecture overview. > > I need Reads to be clarified: > > How memory is used > 1. When data is in the Memtable > 2. When data is in the SSTable > > How cache is used alongside with Memtable? > > Are records created in the Memtable from writes only or from reads also? > > What I need to know is, how Cassandra uses memory and Memtables for reads? > > > Thenk you, > Viktor > --0023547c8bdf759076049cef9c4d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Data is in Memtables from writes before they get flushed (based on first th= reshold of ops/size/time exceeded; all are configurable) to SSTables on dis= k.

There is a keycache and a rowcache.=A0 The keycache caches offset= s into SSTables for the rows.=A0 the rowcache caches the entire row.=A0 The= re is also the OS page cache which is heavily used.

When a read happens, the keycache is updated with the information for t= he SSTables the row was eventually found in.=A0 If there are too many entri= es now in the keycache, some are ejected.=A0 Overall the keycache uses very= little memory per entry and can cut your disk IO in half so it's a pre= tty big win.

If you read an entire row it goes in the row cache.=A0 Like the keycach= e, this may result in older entries being ejected from the cache.=A0 If you= insert lots of really large rows in the rowcache you can OOM your JVM.=A0 = The rowcache is kept up to date with the memtables as writes come in.

When a read comes in, C* will collect the data from the SSTables and Me= mtables and merge them together but data only goes into Memtables from writ= es.

On Tue, Feb 22, 2011 at 3:32 AM, Vikt= or Jevdokimov <vjevdokimov@gmail.com> wrote:
Hello,
=

Write path is perfectly documented in architecture over= view.

I need Reads to be clarified:

= How memory is used
1. When data is in the Memtable
2. When data is in the S= STable

How cache is used alongside with Memtable?<= /div>

Are records created in the Memtable from writes on= ly or from reads also?

What I need to know is, how Cassandra uses memory and M= emtables for reads?


Thenk you,
Viktor

--0023547c8bdf759076049cef9c4d--