hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: BlockCache for large scans.
Date Wed, 09 Apr 2014 21:44:29 GMT
Didn't quite get what you mean, Asaf.

If you're talking about HBASE-5349, please read release note of HBASE-5349.

By default, memstore min/max range is initialized to memstore percent:

    globalMemStorePercentMinRange = conf.getFloat(
MEMSTORE_SIZE_MIN_RANGE_KEY,

        globalMemStorePercent);

    globalMemStorePercentMaxRange = conf.getFloat(
MEMSTORE_SIZE_MAX_RANGE_KEY,

        globalMemStorePercent);

Cheers


On Wed, Apr 9, 2014 at 3:17 PM, Asaf Mesika <asaf.mesika@gmail.com> wrote:

> The Jira says it's enabled by auto. Is there an official explaining this
> feature?
>
> On Wednesday, April 9, 2014, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > Please take a look at http://www.n10k.com/blog/blockcache-101/
> >
> > For D, hbase.regionserver.global.memstore.size is specified in terms of
> > percentage of heap. Unless you enable HBASE-5349 'Automagically tweak
> > global memstore and block cache sizes based on workload'
> >
> >
> > On Wed, Apr 9, 2014 at 12:24 AM, gortiz <gortiz@pragsis.com<javascript:;>>
> > wrote:
> >
> > > I've been reading the book definitive guide and hbase in action a
> little.
> > > I found this question from Cloudera that I'm not sure after looking
> some
> > > benchmarks and documentations from HBase. Could someone explain me a
> > little
> > > about? . I think that when you do a large scan you should disable the
> > > blockcache becuase the blocks are going to swat a lot, so you didn't
> get
> > > anything from cache, I guess you should be penalized since you're
> > spending
> > > memory, calling GC and CPU with this task.
> > >
> > > *You want to do a full table scan on your data. You decide to disable
> > > block caching to see if this**
> > > **improves scan performance. Will disabling block caching improve scan
> > > performance?*
> > >
> > > A.
> > > No. Disabling block caching does not improve scan performance.
> > >
> > > B.
> > > Yes. When you disable block caching, you free up that memory for other
> > > operations. With a full
> > > table scan, you cannot take advantage of block caching anyway because
> > your
> > > entire table won't fit
> > > into cache.
> > >
> > > C.
> > > No. If you disable block caching, HBase must read each block index from
> > > disk for each scan,
> > > thereby decreasing scan performance.
> > >
> > > D.
> > > Yes. When you disable block caching, you free up memory for MemStore,
> > > which improves,
> > > scan performance.
> > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message