Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 78010 invoked from network); 18 Feb 2011 08:26:07 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 18 Feb 2011 08:26:07 -0000 Received: (qmail 66885 invoked by uid 500); 18 Feb 2011 08:26:05 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 66749 invoked by uid 500); 18 Feb 2011 08:26:02 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 66735 invoked by uid 99); 18 Feb 2011 08:26:02 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 18 Feb 2011 08:26:02 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (nike.apache.org: local policy) Received: from [209.85.161.172] (HELO mail-gx0-f172.google.com) (209.85.161.172) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 18 Feb 2011 08:25:52 +0000 Received: by gxk27 with SMTP id 27so1506469gxk.31 for ; Fri, 18 Feb 2011 00:25:28 -0800 (PST) MIME-Version: 1.0 Received: by 10.150.145.9 with SMTP id s9mr721124ybd.125.1298017528471; Fri, 18 Feb 2011 00:25:28 -0800 (PST) Sender: scode@scode.org Received: by 10.150.148.20 with HTTP; Fri, 18 Feb 2011 00:25:28 -0800 (PST) X-Originating-IP: [95.195.117.27] In-Reply-To: References: <3ee12311-35e2-6a66-e7ad-a91b5b61b798@me.com> Date: Fri, 18 Feb 2011 09:25:28 +0100 X-Google-Sender-Auth: kEh01fBag42t7_E4JgqdNBzGP1Q Message-ID: Subject: Re: memory consuption From: Peter Schuller To: user@cassandra.apache.org Cc: Aaron Morton Content-Type: text/plain; charset=UTF-8 X-Virus-Checked: Checked by ClamAV on apache.org > main argument for using mmap() instead of standard I/O is the fact > that reading entails just touching memory - in the case of the memory > being resident, you just read it - you don't even take a page fault > (so no overhead in entering the kernel and doing a semi-context > switch). Oh and in the case of Java/Cassandra, as Jonathan clued me in on earlier, there is also the issue that byte[] arrays are mandated to be zeroed when allocated which causes overhead typically because there has to be a loop[1] somewhere writing a bunch of zeroes in, that you're then just going to replace immediately. Mapping a file has no such implications as long as you read directly from the underlying direct ByteBuffer. [1] Not *necessarily*; a JVM could theoretically do byte[] allocations in such a way that it already knows the contents is zeroed, but it would be highly dependent on the GC/memory management technique used by the JVM whether this is practical. (It just occurred to me that Azul should get this for 'free' in their GC. Wonder if that's true.) -- / Peter Schuller