hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: keyvalue cache
Date Wed, 04 Apr 2012 18:55:54 GMT
On Wed, Apr 4, 2012 at 11:40 AM, Matt Corgan <mcorgan@hotpads.com> wrote:
> I guess the benefit of the KV cache is that you are not holding entire 64K
> blocks in memory when you only care about 200 bytes of them.  Would an
> alternative be to set a small block size (2KB or less)?
> The problems with small block sizes would be expensive block cache
> management overhead and inefficient scanning IO due to lack of read-ahead.
>  Maybe improving the cache management and read-ahead would be more general
> improvements that don't add as much complexity?

I tend to think that there would be bigger bang for the buck doing
such as Matt describes above (plus things like the Todd started
MemStore improvements).

> I'm having a hard time envisioning how you would do invalidations on the KV
> cache and how you would merge its entries into a scan, etc.  Would it
> basically be a memstore in front of the memstore where KVs get individually
> invalidated instead of bulk-flushed?  Would it be sorted or hashed?

In the distant past, ruminations on a KV cache had it that it'd be
hard to do.  I could see it being good for hot cells or short, hot
rows -- it could make some some sub-ms savings I'd guess w/ some
savings in cpu -- for sure but it couldn't really be used by scanners
(least not w/o some interesting gymnastics).


View raw message