Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of kevin.odell@cloudera.com
 designates 209.85.215.48 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAN89x=wYQ6bgBJoh7ZG+=y1DZmD1CU_VTw_SWN4KqDJ6BfMSqw@mail.gmail.com>
References: 
 <CAN89x=wYQ6bgBJoh7ZG+=y1DZmD1CU_VTw_SWN4KqDJ6BfMSqw@mail.gmail.com>
Date: Mon, 8 Jul 2013 14:25:50 -0400
Message-ID: 
 <CAGngS9f5=ApEvgC=5LgD-RuF_n1Y8XYkai3wDm-w0DAWaCwGHA@mail.gmail.com>
Subject: Re: GC recommendations for large Region Server heaps
From: "Kevin O'dell" <kevin.odell@cloudera.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a11c3b4ae932e1304e1042c6c

--001a11c3b4ae932e1304e1042c6c
Content-Type: text/plain; charset=ISO-8859-1

Hey Suraj,

  I would recommend turning on MSlab and using the following settings:

-XX:+UseParNewGC -XX:+UseConcMarkSweepGC
-XX:CMSInitiatingOccupancyFraction=70 -XX:+CMSParallelRemarkEnabled


On Mon, Jul 8, 2013 at 2:09 PM, Suraj Varma <svarma.ng@gmail.com> wrote:

> Hello:
> We have an HBase cluster with region servers running on 8GB heap size with
> a 0.6 block cache (it is a read heavy cluster, with bursty write traffic
> via MR jobs). (version: hbase-0.94.6.1)
>
> During HBaseCon, while speaking to a few attendees, I heard some folks were
> running region servers as high as 24GB and some others in the 16GB range.
>
> So - question: Are there any special GC recommendations (tuning parameters,
> flags, etc) that folks who run at these large heaps can recommend while
> moving up from an 8GB heap? i.e. for 16GB and for 24GB RS heaps ... ?
>
> I'm especially concerned about long pauses causing zk session timeouts and
> consequent RS shutdowns. Our boxes do have a lot of RAM and we are
> exploring how we can use more of it for the cluster while maintaining
> overall stability.
>
> Also - if there are clusters running multiple region servers per host, I'd
> be very interested to know what RS heap sizes those are being run at ...
> and whether this was chosen as an alternative to running a single RS with
> large heap.
>
> (I know I'll have to test the GC stuff out on my cluster and for my
> workloads anyway ... but just trying to get a feel of what sort of tuning
> options had to be used to have a stable HBase cluster with 16 or 24GB RS
> heaps).
>
> Thanks in advance,
> --Suraj
>


-- 
Kevin O'Dell
Systems Engineer, Cloudera

--001a11c3b4ae932e1304e1042c6c--