Return-Path: Delivered-To: apmail-hbase-user-archive@www.apache.org Received: (qmail 74931 invoked from network); 3 Jan 2011 22:05:32 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 3 Jan 2011 22:05:32 -0000 Received: (qmail 61110 invoked by uid 500); 3 Jan 2011 22:05:31 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 61077 invoked by uid 500); 3 Jan 2011 22:05:31 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 61069 invoked by uid 99); 3 Jan 2011 22:05:31 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jan 2011 22:05:31 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,RFC_ABUSE_POST,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of saint.ack@gmail.com designates 209.85.161.41 as permitted sender) Received: from [209.85.161.41] (HELO mail-fx0-f41.google.com) (209.85.161.41) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jan 2011 22:05:23 +0000 Received: by fxm12 with SMTP id 12so7316415fxm.14 for ; Mon, 03 Jan 2011 14:05:03 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:sender:received :in-reply-to:references:date:x-google-sender-auth:message-id:subject :from:to:content-type; bh=5KhoIwlxgwfqf5q9MTQaAiQ4/xZ3wOjgz/it4I0rHWk=; b=iVzCLm08FOSzZHRoJk89kxPH4/dCeSRPmHSlhL2aRJF/Hwxzx36xYwR0RiUB0+s7bE B6q9xe5cf/K7+ypW/cKeIx/96D9FRLMvgoywXVdImJZ2IML/UlElFgUtKNKn0vnsrafZ 8TK/lViEwy4OPQYPh1m2fQXVx+V9zrdnMwKic= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:content-type; b=Ocfbs4axfOdMV23HknmFgaRrce3RTyYAAeqw9AtbDtb7E1mN8wFOfnTvKlZ7O18vuR K4ZaMHddRbW1V6L0HiLjaMyfyKn2pHdiDkWMb+jB2a0uhCgg5EJpnTMju1J/lYF/bEoW LSjG++psX9CQy5327aPEn9ICYq3I+6xkGatSo= MIME-Version: 1.0 Received: by 10.223.97.7 with SMTP id j7mr501730fan.18.1294092303394; Mon, 03 Jan 2011 14:05:03 -0800 (PST) Sender: saint.ack@gmail.com Received: by 10.223.83.136 with HTTP; Mon, 3 Jan 2011 14:05:03 -0800 (PST) In-Reply-To: References: Date: Mon, 3 Jan 2011 14:05:03 -0800 X-Google-Sender-Auth: GBThyd8UpIs7IOZ4m6cc7tVYcxI Message-ID: Subject: Re: CMF & NodeIsDeadException From: Stack To: user@hbase.apache.org Content-Type: text/plain; charset=ISO-8859-1 X-Virus-Checked: Checked by ClamAV on apache.org On Mon, Jan 3, 2011 at 12:50 PM, Wayne wrote: > We have an 8GB heap. What should newsize be? I just had another node die > hard after going into a CMF storm. I swear it had solid CMFs 30+ in a row. > Did a full stop-the-world GC run in between? It should have cleaned up fragmentation. > I have no idea what eden space is or how to see what it is. ?? > Sorry. There's a bunch of 'cute' terms used for describing the two heap areas in the JVM. Basically, new stuff goes into the 'new' or 'eden' area first. If it sticks around through N (configurable) GCs, it gets promoted to old or tenured generation (there are other names for these notions of young and old). The garbage collection algorithms done in the two heaps differ. See the Ted citation for more on the gruesome details (though come up to a newer version of that doc). The JVM is supposed to work ergonomically but it just ain't smart enough dealing w/ HBase/Cass loadings it seems (e.g. it keeps growing the new/eden space pathologically it would seem). > Not knowing what else to do I will start using some of the Cassandra > settings I used to improve it by setting the occupancy fraction. Any other > ideas??? > Which config. you talking of? -XX:+CMSInitiatingOccupancyFraction? Thats a good one to toggle down from defaullts. Should help put off promotion failures a while. St.Ack