hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Beaudreault <bbeaudrea...@hubspot.com>
Subject Re: hbase block-cache scan.setCaching(false) not being respected
Date Mon, 02 Jun 2014 20:05:54 GMT
The Block Cache is used for more than just the scanner caching.
 Additionally, *hfile.block.cache.size *is a server-side config, while
scan.setCaching(false) is on an RPC-level.  So regardless of your
setCaching value the RegionServers will continue to allocate memory to the
block cache.

Check out
http://hbase.apache.org/book/regionserver.arch.html#block.cache.usage for
more details, specifically it lists other residents of the block cache.

On Mon, Jun 2, 2014 at 3:55 PM, Matt K <matvey1414@gmail.com> wrote:

> Hi all,
> We are running a number of Map/Reduce jobs on top of HBase. We are not
> using HBase for any of its realtime capabilities, only for
> batch-processing. So we aren't doing lookups, just scans.
> Each one of our jobs has *scan.setCaching(false)* to turn off
> block-caching, since each block will only be accessed once.
> We recently started using Cloudera Manager, and I’m seeing something that
> doesn’t add up. See image below. It’s clear from the graphs that Block
> Cache is being used currently, and blocks are being cached and evicted.
> We do have *hfile.block.cache.size* set to 0.4 (default), but my
> understanding is that the jobs setting scan.setCaching(false) should
> override this. Since it’s set in every job, there should be no blocks being
> cached.
> Can anyone help me understand what we’re seeing?
> Thanks,
> -Matt
> [image: Inline image 1]

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message