Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0BE68960D for ; Sat, 10 Mar 2012 22:38:42 +0000 (UTC) Received: (qmail 84580 invoked by uid 500); 10 Mar 2012 22:38:40 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 84558 invoked by uid 500); 10 Mar 2012 22:38:40 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 84550 invoked by uid 99); 10 Mar 2012 22:38:40 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 10 Mar 2012 22:38:40 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of edlinuxguru@gmail.com designates 209.85.210.172 as permitted sender) Received: from [209.85.210.172] (HELO mail-iy0-f172.google.com) (209.85.210.172) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 10 Mar 2012 22:38:35 +0000 Received: by iazz13 with SMTP id z13so4787195iaz.31 for ; Sat, 10 Mar 2012 14:38:15 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=R6CM9nuJVegoBomMkiwc6e0Xsce0jVNtm3ipy0vBGbc=; b=P0D/dbqb4fnMbX5ujsGkhnt4BGufNBfJA04nkp0Wzs9fCryi2MMpYOkE3FXoz3a7tc Tha3m7IQOCgExrH1mqf86N5GXNnqKBUyMG4IEbJ4kYWv44fefvtQy8HQbOQO+ZAqDH6a U4QVuwfOLTw7n4raV+OtSQwKNwR/EoTD4377JE1+LfMjkFh6FiV7hcESnPVHTeI9z/Wg w1EM45IEN6fsGjEsy9a7MK4QIe7Vk0REGKg/PHFHwS+78P6kZjEAuQrEvNsab23FDaKE qXXbNdFi3yJZlkNSrg6y0t5nmK8qNr1N25bbPnEYccjYxzmm11rujgUfXHeWYoPOo76s B36A== MIME-Version: 1.0 Received: by 10.50.203.100 with SMTP id kp4mr11472059igc.6.1331419095181; Sat, 10 Mar 2012 14:38:15 -0800 (PST) Received: by 10.42.197.1 with HTTP; Sat, 10 Mar 2012 14:38:15 -0800 (PST) In-Reply-To: References: Date: Sat, 10 Mar 2012 17:38:15 -0500 Message-ID: Subject: Re: LeveledCompaction and/or SnappyCompressor causing memory pressure during repair From: Edward Capriolo To: user@cassandra.apache.org Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org The only downside of compression is it does cause more memory pressure. I can imagine something like repair could confound this. Since it would seem like building the merkle tree would involve decompressing every block on disk. I have been attempting to determine if the block size being larger or smaller has any effect on memory pressure. On Sat, Mar 10, 2012 at 4:50 PM, Peter Schuller wrote: >> However, when I run a repair my CMS usage graph no longer shows sudden drops >> but rather gradual slopes and only manages to clear around 300MB each GC. >> This seems to occur on 2 other nodes in my cluster around the same time, I >> assume this is because they're the replicas (we use 3 replicas). Parnew >> collections look about the same on my graphs with or without repair running >> so no trouble there so far as I can tell. > > I don't know why leveled/snappy would affect it, but disregarding > that, I would have been suggesting that you are seeing additional heap > usage because of long-running repairs retaining sstables and delaying > their unload/removal (index sampling/bloom filters filling your heap). > If it really only happens for leveled/snappy however, I don't know > what that might be caused by. > > -- > / Peter Schuller (@scode, http://worldmodscode.wordpress.com)