Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of varun@pinterest.com
 designates 209.85.223.173 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <1358461899.37778.YahooMailNeo@web140603.mail.bf1.yahoo.com>
References: 
 <CAKxWWm11N+EF930nRK46gos8BeBe6e4uoG63S77F57_uFCiECQ@mail.gmail.com>
	<1358461899.37778.YahooMailNeo@web140603.mail.bf1.yahoo.com>
Date: Thu, 17 Jan 2013 15:24:14 -0800
Message-ID: 
 <CAKxWWm24F0iZbMBLWbkZp+AqmjXmoAa-J8HpJyYMdKoGeTkNtA@mail.gmail.com>
Subject: Re: Hbase heap size
From: Varun Sharma <varun@pinterest.com>
To: user@hbase.apache.org, lars hofhansl <larsh@apache.org>
Content-Type: multipart/alternative; boundary=e89a8f234ab10395e604d3844bf3

--e89a8f234ab10395e604d3844bf3
Content-Type: text/plain; charset=ISO-8859-1

Thanks for the info. I am looking for a balance where I have a write heavy
work load and need excellent read latency. So 40 % to block cache for
caching, 35 % to memstore.

But I would like to also reduce the number of HFiles and amount of
compaction activity. So, having few number of regions and much larger
memstore flush size - like 640M. Could a large memstore flush be a problem
in some sense ? Are updates blocked on memstore flush ? In my case, I would
expect a 600M sized memstore to materialize into a 200-300M sized HFile.

On Thu, Jan 17, 2013 at 2:31 PM, lars hofhansl <larsh@apache.org> wrote:

> A good rule of thumb that I found is to give each region server a Java
> help that is roughly 1/100th of the size of the disk space per region
> server.
> (that is assuming all the default setting: 10G regions, 128M memstores,
> 40% of heap for memstores, 20% of heap for block cache, 3-way replication)
>
>
> That is, if you give the region server a 10G heap, you can expect to be
> able to serve about 1T worth of disk space.
>
> That can be tweaked of course (increase the region size to 20G, if your
> load is mostly readonly you shrink the memstores, etc).
> That way you can reduce that ratio to 1/200 or even less.
>
>
> I'm sure other folks will have more detailed input.
>
>
> -- Lars
>
>
>
> ________________________________
>  From: Varun Sharma <varun@pinterest.com>
> To: user@hbase.apache.org
> Sent: Thursday, January 17, 2013 1:15 PM
> Subject: Hbase heap size
>
> Hi,
>
> I was wondering how much folks typical give to hbase and how much they
> leave for the file system cache for the region server. I am using hbase
> 0.94 and running only the region server and data node daemons. I have a
> system with 15G ram.
>
> Thanks
>

--e89a8f234ab10395e604d3844bf3--