Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of bbeaudreault@hubspot.com
 designates 74.125.150.108 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAGGEs2ziD3AKzvQNyoc_SAY7wGWWSeAkaoS8dfLPqS0+ycb45A@mail.gmail.com>
References: 
 <CAGGEs2ziD3AKzvQNyoc_SAY7wGWWSeAkaoS8dfLPqS0+ycb45A@mail.gmail.com>
From: Bryan Beaudreault <bbeaudreault@hubspot.com>
Date: Mon, 2 Jun 2014 16:05:54 -0400
Message-ID: 
 <CANZDn9tq9NORXdSM2DyYhmTH8bNUPanNF46eeo9aQ=2yL5t0wA@mail.gmail.com>
Subject: Re: hbase block-cache scan.setCaching(false) not being respected
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=001a11335a8e660edb04fadfed13

--001a11335a8e660edb04fadfed13
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

The Block Cache is used for more than just the scanner caching.
 Additionally, *hfile.block.cache.size *is a server-side config, while
scan.setCaching(false) is on an RPC-level.  So regardless of your
setCaching value the RegionServers will continue to allocate memory to the
block cache.

Check out
http://hbase.apache.org/book/regionserver.arch.html#block.cache.usage for
more details, specifically it lists other residents of the block cache.


On Mon, Jun 2, 2014 at 3:55 PM, Matt K <matvey1414@gmail.com> wrote:

> Hi all,
>
> We are running a number of Map/Reduce jobs on top of HBase. We are not
> using HBase for any of its realtime capabilities, only for
> batch-processing. So we aren't doing lookups, just scans.
>
> Each one of our jobs has *scan.setCaching(false)* to turn off
> block-caching, since each block will only be accessed once.
>
> We recently started using Cloudera Manager, and I=E2=80=99m seeing someth=
ing that
> doesn=E2=80=99t add up. See image below. It=E2=80=99s clear from the grap=
hs that Block
> Cache is being used currently, and blocks are being cached and evicted.
>
> We do have *hfile.block.cache.size* set to 0.4 (default), but my
> understanding is that the jobs setting scan.setCaching(false) should
> override this. Since it=E2=80=99s set in every job, there should be no bl=
ocks being
> cached.
>
> Can anyone help me understand what we=E2=80=99re seeing?
>
> Thanks,
>
> -Matt
>
> [image: Inline image 1]
>

--001a11335a8e660edb04fadfed13--