Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of saint.ack@gmail.com designates
 209.85.212.41 as permitted sender)
MIME-Version: 1.0
Sender: saint.ack@gmail.com
In-Reply-To: 
 <CAOKsKJVNY4Ngq3rwhX4R7WjiDzDQGe4b0oz=uG8rgzY+cj+-fA@mail.gmail.com>
References: 
 <CAMUu0w91b8Ki1BaUxLAymqf=o2v0ejjb-rYcjTETYij4jO6Gcw@mail.gmail.com>
	<DC5EBE7F3610EB4CA5C7E92D76873E151716F05630@exchange2007.carrieriq.com>
	<CAMUu0w-UwA9FYvqoCj=GOsN-ECL1=88op4JDo0iY3afOmou=oA@mail.gmail.com>
	<CAOKsKJVNY4Ngq3rwhX4R7WjiDzDQGe4b0oz=uG8rgzY+cj+-fA@mail.gmail.com>
Date: Wed, 4 Apr 2012 11:55:54 -0700
Message-ID: 
 <CADcMMgEXhFkXxP71kFUMjGMf2_RpTSje8zZYEkRDm3d1Z25S6w@mail.gmail.com>
Subject: Re: keyvalue cache
From: Stack <stack@duboce.net>
To: dev@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

On Wed, Apr 4, 2012 at 11:40 AM, Matt Corgan <mcorgan@hotpads.com> wrote:
> I guess the benefit of the KV cache is that you are not holding entire 64=
K
> blocks in memory when you only care about 200 bytes of them. =A0Would an
> alternative be to set a small block size (2KB or less)?
>
> The problems with small block sizes would be expensive block cache
> management overhead and inefficient scanning IO due to lack of read-ahead=
.
> =A0Maybe improving the cache management and read-ahead would be more gene=
ral
> improvements that don't add as much complexity?
>

I tend to think that there would be bigger bang for the buck doing
such as Matt describes above (plus things like the Todd started
MemStore improvements).

> I'm having a hard time envisioning how you would do invalidations on the =
KV
> cache and how you would merge its entries into a scan, etc. =A0Would it
> basically be a memstore in front of the memstore where KVs get individual=
ly
> invalidated instead of bulk-flushed? =A0Would it be sorted or hashed?
>

In the distant past, ruminations on a KV cache had it that it'd be
hard to do.  I could see it being good for hot cells or short, hot
rows -- it could make some some sub-ms savings I'd guess w/ some
savings in cpu -- for sure but it couldn't really be used by scanners
(least not w/o some interesting gymnastics).

St.Ack