Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id A69F11104E for ; Tue, 29 Jul 2014 15:49:40 +0000 (UTC) Received: (qmail 8922 invoked by uid 500); 29 Jul 2014 15:49:40 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 8894 invoked by uid 500); 29 Jul 2014 15:49:40 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 8878 invoked by uid 99); 29 Jul 2014 15:49:40 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Jul 2014 15:49:40 +0000 Date: Tue, 29 Jul 2014 15:49:40 +0000 (UTC) From: "Michael Shuler (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CASSANDRA-7467) flood of "setting live ratio to maximum of 64" from repair MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-7467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14077859#comment-14077859 ] Michael Shuler commented on CASSANDRA-7467: ------------------------------------------- [~enigmacurry] could you see if this can be reproduced? Thanks! > flood of "setting live ratio to maximum of 64" from repair > ---------------------------------------------------------- > > Key: CASSANDRA-7467 > URL: https://issues.apache.org/jira/browse/CASSANDRA-7467 > Project: Cassandra > Issue Type: Bug > Components: Core > Reporter: Jackson Chung > Fix For: 2.0.10 > > > we are on 2.0.8 > running with repair -pr -local , all nodes on i2.2x (60G ram);, with setting 8G of heap. Using java 8. (key cache size is 1G) > On occasion, when repair is run, the C* that run the repair, or another node in the cluster, or both, run into a bad state with the system.log just printing ""setting live ratio to maximum of 64" forever every split seconds. It usually happens when repairing one of the larger/wider CF. > WARN [MemoryMeter:1] 2014-06-28 09:13:24,540 Memtable.java (line 470) setting live ratio to maximum of 64.0 instead of Infinity > INFO [MemoryMeter:1] 2014-06-28 09:13:24,540 Memtable.java (line 481) CFS(Keyspace='RIQ', ColumnFamily='MemberTimeline') liveRatio is 64.0 (just-counted was 64.0). calculation took 0ms for 0 cells > Table: MemberTimeline > SSTable count: 13 > Space used (live), bytes: 17644018786 > ... > Compacted partition minimum bytes: 30 > Compacted partition maximum bytes: 464228842 > Compacted partition mean bytes: 54578 > Just to give an idea of how bad this is, the log file is set to rotate 50 times with 21M each. In less than 15 minutes, all the logs are filled up with just that log. C* is not responding, and can't be killed normally. Only way is to kill -9 -- This message was sent by Atlassian JIRA (v6.2#6252)