hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@gmail.com>
Subject Re: hbase.bucketcache.percentage.in.combinedcache with ioengine=<FILE ON SSD> on HBase 0.98
Date Wed, 07 Oct 2015 20:55:19 GMT
I compared "offheap" vs file:tmpfs in the "showdown" blog post I linked
previously. For that test, they were comparable.
Configuring MaxDirectMemory is only required for "offheap", not the other
configurations. In fact, if you use both tmpfs and MaxDirectMemory, you'll
probably run out of memory very quickly (or start swapping?) as JVM tries
to allocate the memory you already mounted to tmpfs.

On Tue, Oct 6, 2015 at 1:29 PM, donmai <doodnet@gmail.com> wrote:

> Thanks Nick! Unfortunately, I'm stuck with 0.98. Perhaps it's something I
> should look into backporting if all else fails...
>
> Would also bigger great if anyone had any insight in the differences
> between offheap and file:/ in terms of configuration. When RAM to tmpfs was
> used as the workaround for the RS issue, was MaxDirectMemory size still
> set?
>
> Thanks!
> On Oct 6, 2015 4:03 PM, "Nick Dimiduk" <ndimiduk@apache.org> wrote:
>
> > Configuring offheap cache in 0.98 is a huge PITA. There are some example
> > configs [0] linked from my old blog post [1], maybe that's a starting
> > point? I also worked with our docs team to get this explained [2] in a
> > somewhat useful way, using an algorithm/formula as the basis of the
> > explanation. That algorithm was eventually implemented into Ambari's
> Stack
> > Advisor script [3], so if you're really feeling brave you can try to
> > extract it from there.
> >
> > But really, you should upgrade to 1.x and enjoy Stack's nice work on
> making
> > it simpler to configure.
> >
> > Good luck,
> > -n
> >
> > [0]: https://github.com/ndimiduk/perf_blockcache
> > [1]: http://www.n10k.com/blog/blockcache-showdown/
> > [2]:
> >
> >
> http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_Sys_Admin_Guides/content/ref-db219cd6-c586-49c1-bc56-c9c1c5475276.1.html
> > [3]:
> >
> >
> https://github.com/apache/ambari/blob/trunk/ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py#L552-L579
> >
> > On Tue, Oct 6, 2015 at 11:36 AM, Anthony Nguyen <
> > anthony.an.nguyen@gmail.com
> > > wrote:
> >
> > > Hi all,
> > >
> > > I'm running HBase 0.98, with the complicated
> percentage.in.combinedcache
> > > setting. What's the best way to set the percentage.in.combinedcache
> > setting
> > > when using a file on SSD storage as the location for ioengine? The SSD
> > > drive is dedicated for the cache. I don't want to modify the
> > LruBlockCache
> > > settings at all, keeping it available for index + bloom and just add
> the
> > L2
> > > SSD cache for storage for the data blocks (not the raw L1+L2 method).
> > >
> > > My confusion is in how I should best modify the
> > percentage.in.combinedcache
> > > setting. HBASE-11520 makes things much clearer, but unfortunately it
> > isn't
> > > in 0.98. To me, it doesn't make much sense to have L1 be set to
> > > hbase.bucketcache.size (100G) * (1.0 -
> > > hbase.bucketcache.percentage.in.combinedcache (default=0.9) = 1G when I
> > > want the L1 to be the same as it was prior to bucketcache
> > > (hfile.block.cache.size * heap).
> > >
> > > Also, I don't want to have any direct addressing of memory since my
> > offheap
> > > is completely on SSD. Does this mean I shouldn't change
> > > XX:MaxDirectMemorySize?
> > >
> > > Could someone sanity check these settings for me? Thanks!
> > >
> > > 30 GB RAM
> > > ~120 GB SSD
> > > HBASE_REGIONSERVER_OPTS -Xms17G -Xmx17G
> > > XX:MaxDirectMemorySize <- Don't touch since we're not using direct byte
> > > buffers, right?
> > >
> > > <property>
> > >   <name>hbase.bucketcache.ioengine</name>
> > >   <value>file:/mnt/foo/bar</value>
> > > </property>
> > > <property>
> > >   <name>hbase.bucketcache.percentage.in.combinedcache</name>
> > >   <value>0.9</value>
> > > </property>
> > >
> > > <property>
> > >   <name>hfile.block.cache.size</name>
> > >   <value>0.6</value>
> > > </property>
> > >
> > > <property>
> > >   <name>hbase.bucketcache.size</name>
> > >   <value>102400</value>
> > > </property>
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message