Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of matvey1414@gmail.com designates
 209.85.220.179 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CALte62z7=4pHfL8zefB1Wz+noTwAomT67shT5d1KDeV0B-jJBQ@mail.gmail.com>
References: 
 <CAGGEs2ziD3AKzvQNyoc_SAY7wGWWSeAkaoS8dfLPqS0+ycb45A@mail.gmail.com>
	<CALte62z7=4pHfL8zefB1Wz+noTwAomT67shT5d1KDeV0B-jJBQ@mail.gmail.com>
Date: Mon, 2 Jun 2014 19:02:18 -0400
Message-ID: 
 <CAGGEs2y_KT-+GkRW2SVyh9dxyiEk0sMQw5UEcx0ohKYxrnKNiQ@mail.gmail.com>
Subject: Re: hbase block-cache scan.setCaching(false) not being respected
From: Matt K <matvey1414@gmail.com>
To: user@hbase.apache.org
Content-Type: multipart/alternative; boundary=20cf3033471515075d04fae263b8

--20cf3033471515075d04fae263b8
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi, thanks for responses.

Ted - when I said "scan.setCaching", I meant "scan.setCacheBlocks(false)".
That's what I get for not copying/pasting directly from code :)

I added a link to the graphs here:
https://drive.google.com/file/d/0B3ZQ0nMNMFxCOHZNZVFsWEhCOUU/edit?usp=3Dsha=
ring

Bryan - I believe you're right, but wanted to confirm.

Thanks,
-Matt


On Mon, Jun 2, 2014 at 4:09 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Have you added the following when passing Scan to your job ?
>
> scan.setCacheBlocks(false);
>
> BTW image didn't go through.
> Consider putting image on third-party site.
>
> On Mon, Jun 2, 2014 at 12:55 PM, Matt K <matvey1414@gmail.com> wrote:
>
> > Hi all,
> >
> > We are running a number of Map/Reduce jobs on top of HBase. We are not
> > using HBase for any of its realtime capabilities, only for
> > batch-processing. So we aren't doing lookups, just scans.
> >
> > Each one of our jobs has *scan.setCaching(false)* to turn off
> > block-caching, since each block will only be accessed once.
> >
> > We recently started using Cloudera Manager, and I=E2=80=99m seeing some=
thing that
> > doesn=E2=80=99t add up. See image below. It=E2=80=99s clear from the gr=
aphs that Block
> > Cache is being used currently, and blocks are being cached and evicted.
> >
> > We do have *hfile.block.cache.size* set to 0.4 (default), but my
> > understanding is that the jobs setting scan.setCaching(false) should
> > override this. Since it=E2=80=99s set in every job, there should be no =
blocks
> being
> > cached.
> >
> > Can anyone help me understand what we=E2=80=99re seeing?
> >
> > Thanks,
> >
> > -Matt
> >
> > [image: Inline image 1]
> >
>


--=20
www.calcmachine.com - easy online calculator.

--20cf3033471515075d04fae263b8--