Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5C47B6614 for ; Fri, 15 Jul 2011 16:10:30 +0000 (UTC) Received: (qmail 12018 invoked by uid 500); 15 Jul 2011 16:10:28 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 11951 invoked by uid 500); 15 Jul 2011 16:10:27 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 11943 invoked by uid 99); 15 Jul 2011 16:10:27 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 Jul 2011 16:10:27 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.212.44] (HELO mail-vw0-f44.google.com) (209.85.212.44) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 15 Jul 2011 16:10:19 +0000 Received: by vws12 with SMTP id 12so1411232vws.31 for ; Fri, 15 Jul 2011 09:09:57 -0700 (PDT) MIME-Version: 1.0 Received: by 10.52.177.36 with SMTP id cn4mr2691280vdc.26.1310746197538; Fri, 15 Jul 2011 09:09:57 -0700 (PDT) Sender: scode@scode.org Received: by 10.52.166.136 with HTTP; Fri, 15 Jul 2011 09:09:57 -0700 (PDT) X-Originating-IP: [94.234.170.40] In-Reply-To: References: Date: Fri, 15 Jul 2011 18:09:57 +0200 X-Google-Sender-Auth: fVbke9UmEHPZvh9ZDu_04RZNnhY Message-ID: Subject: Re: Cache layer in front of cassandra... any help / suggestions? From: Peter Schuller To: user@cassandra.apache.org, sumanthewhiz@gmail.com Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable > As we work on telecom data records (voice call/sms/GPRS xDRs), the data > volume is simply HUGE, and we definitely need a =E2=80=9Ccontrolled=E2=80= =9D caching > mechanism in front of the Cassandra layer. What's huge? Number of gigs, ballpark. > By the term =C2=A0=E2=80=9Ccontrolled cache layer=E2=80=9D, what I am try= ing to suggest is > something like maybe maintaining a list of most high-usage (and therefore= , > high occurrence) phone numbers somewhere, and the cache layer will hold a= ll > live data and counters for those numbers in memory. Therefore, all The cassandra row-cache is LRU, and the page cache of OS:es is "LRU:ish" (but generally you might see evictions at any time when unlucky). If you use an external cache, keep in mind that you instantly have the problem that the cache can become inconsistent with data in Cassandra. You may also want to wait for the off-heap row cache support to be in a released version to be more efficient w.r.t. memory usage and GC overhead than the normal row caching behavior. But before asking what the appropriate external cache is, make sure you actually do need one first since the lack of guaranteed consistency with the Cassandra cluster is usually something that is nice to avoid. --=20 / Peter Schuller (@scode on twitter)