From user-return-30483-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Thu Dec 6 02:50:03 2012 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id DE68CD5E2 for ; Thu, 6 Dec 2012 02:50:03 +0000 (UTC) Received: (qmail 92864 invoked by uid 500); 6 Dec 2012 02:50:01 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 92826 invoked by uid 500); 6 Dec 2012 02:50:01 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 92805 invoked by uid 99); 6 Dec 2012 02:50:00 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Dec 2012 02:50:00 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a47.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Dec 2012 02:49:52 +0000 Received: from homiemail-a47.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a47.g.dreamhost.com (Postfix) with ESMTP id AA05D28406E for ; Wed, 5 Dec 2012 18:49:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=from :content-type:message-id:mime-version:subject:date:references:to :in-reply-to; s=thelastpickle.com; bh=McBQRbnNNOE/S0utNb6++FaySy Q=; b=NQuIphbditMFuLfLpnJ0Wm4vN/tJc/IByz0Th+xCWJRFJNw4r7hqs3OUZM aVARhed0iw/f/GndD3Mn4HtJQyZB8fk/Z1JkiR9cDA2J2Fvl1HV9BOqFxCYcUoSQ fwnSfhnIqOT6moUEKfclWE3h5STkPkDICGNRBjXSWpeGTmp6c= Received: from [192.168.2.13] (unknown [116.90.132.105]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a47.g.dreamhost.com (Postfix) with ESMTPSA id 351DC28405C for ; Wed, 5 Dec 2012 18:49:31 -0800 (PST) From: aaron morton Content-Type: multipart/alternative; boundary="Apple-Mail=_DBA70A7A-9D90-4938-85BA-6D515FC9311A" Message-Id: Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\)) Subject: Re: Help on MMap of SSTables Date: Thu, 6 Dec 2012 15:49:29 +1300 References: <81D05238-BD4C-46C4-AAAA-CD6A68CF7567@thelastpickle.com> To: user@cassandra.apache.org In-Reply-To: X-Mailer: Apple Mail (2.1499) X-Virus-Checked: Checked by ClamAV on apache.org --Apple-Mail=_DBA70A7A-9D90-4938-85BA-6D515FC9311A Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=iso-8859-1 Background http://en.wikipedia.org/wiki/Memory-mapped_file > Is it going to load only relevant pages per SSTable on read or is it = going to load an entire SSTable on first access? It will load what is requested, and maybe some additional data taking = into account the amount of memory available for caches.=20 > Say suppose compaction kicks in. Will it then evict hot MMapped pages = for read and substitute it with a lot of pages involving full SSTables? Some file access in cassandra, such as compaction, hints to the OS that = the reads should not be cached. Technically is uses posix_fadvise if you = want to look it up. Cheers ----------------- Aaron Morton Freelance Cassandra Developer New Zealand @aaronmorton http://www.thelastpickle.com On 5/12/2012, at 11:04 PM, Ravikumar Govindarajan = wrote: > Thanks Aaron, >=20 > I am not quite clear on how MMap loads SSTables other than the fact = that it kicks in only during a first-time access >=20 > Is it going to load only relevant pages per SSTable on read or is it = going to load an entire SSTable on first access? >=20 > Say suppose compaction kicks in. Will it then evict hot MMapped pages = for read and substitute it with a lot of pages involving full SSTables? >=20 > -- > Ravi >=20 > On Wed, Dec 5, 2012 at 1:22 AM, aaron morton = wrote: >> Will MMapping data files be detrimental for reads, in this case? > No.=20 >=20 >> In general, when should we opt for MMap data files and what are the = factors that need special attention when enabling the same? > mmapping is the default, so I would say use it until you have a reason = not to.=20 >=20 > mmapping will map the entire file, but pages of data are read into = memory on demand and purged when space is needed.=20 >=20 > Cheers >=20 > ----------------- > Aaron Morton > Freelance Cassandra Developer > New Zealand >=20 > @aaronmorton > http://www.thelastpickle.com >=20 > On 4/12/2012, at 11:59 PM, Ravikumar Govindarajan = wrote: >=20 >> Our current SSTable sizes are far greater than RAM. {150 Gigs of = data, 32GB RAM}. Currently we run with mlockall and mmap_index_only = options and don't experience swapping at all. >>=20 >> We use wide rows and size-tiered-compaction, so a given key will = definitely be spread across multiple sstables. Will MMapping data files = be detrimental for reads, in this case? >>=20 >> In general, when should we opt for MMap data files and what are the = factors that need special attention when enabling the same? >>=20 >> -- >> Ravi >=20 >=20 --Apple-Mail=_DBA70A7A-9D90-4938-85BA-6D515FC9311A Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=iso-8859-1 http://en.wikiped= ia.org/wiki/Memory-mapped_file

Is it going to load only relevant pages per SSTable on = read or is it going to load an entire SSTable on first = access?
It will load what is requested, and maybe some = additional data taking into account the amount of memory available for = caches. 

Say = suppose compaction kicks in. Will it then evict hot MMapped pages for = read and substitute it with a lot of pages involving full = SSTables?
Some file access in cassandra, such as = compaction, hints to the OS that the reads should not be cached. = Technically is uses posix_fadvise if you want to look it = up.

Cheers


http://www.thelastpickle.com

On 5/12/2012, at 11:04 PM, Ravikumar Govindarajan <ravikumar.govindarajan@gm= ail.com> wrote:

Thanks = Aaron,

I am not quite clear on how MMap loads = SSTables other than the fact that it kicks in only during a first-time = access

Is it going to load only relevant pages = per SSTable on read or is it going to load an entire SSTable on first = access?

Say suppose compaction kicks in. Will it then evict = hot MMapped pages for read and substitute it with a lot of pages = involving full = SSTables?

--
Ravi

On Wed, Dec 5, 2012 at 1:22 AM, aaron morton <aaron@thelastpickle.com> = wrote:
Will MMapping data files be detrimental for reads, in this = case?
No. 

In general, when should we opt for MMap data files and = what are the factors that need special attention when enabling the = same?
mmapping is the default, so I would say use it until you have a = reason not to. 

mmapping will map the entire = file, but pages of data are read into memory on demand and purged when = space is needed. 

Cheers

-----------------
Aaron Morton
Freelance = Cassandra Developer
New = Zealand

@aaronmorton

On 4/12/2012, at 11:59 PM, Ravikumar Govindarajan <ravikumar.govindarajan@gmail.com> = wrote:

Our current SSTable sizes are = far greater than RAM. {150 Gigs of data, 32GB RAM}. Currently we run = with mlockall and mmap_index_only options and don't experience swapping = at all.

We use wide rows and size-tiered-compaction, so a given = key will definitely be spread across multiple sstables. Will MMapping = data files be detrimental for reads, in this case?

In general, when should we opt for MMap data files = and what are the factors that need special attention when enabling the = same?

--
Ravi
=


= --Apple-Mail=_DBA70A7A-9D90-4938-85BA-6D515FC9311A--