hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Dimiduk <ndimi...@gmail.com>
Subject Re: blockcache 101
Date Tue, 15 Apr 2014 17:13:00 GMT
On Mon, Apr 14, 2014 at 10:12 PM, Todd Lipcon <todd@cloudera.com> wrote:

> Hmm... in "v2.pdf" here you're looking at different ratios of DB size
> to cache size, but there's also the secondary cache on the system (the
> OS block cache), right?

Yes, this is true.

So when you say only 20GB "memory under management", in fact you're still
> probably getting 100% hit rate on the case where the DB is bigger than RAM,
> right?

I can speculate, likely that's true, but I don't know this for certain. At
the moment, the only points of instrumentation in the harness are in the
HBase client. The next steps include pushing down into the RS, DN and
further to then is to the OS itself.

Maybe would be better to have each graph show the different cache
> implementations overlaid, rather than the different ratios overlaid? That
> would better differentiate the scaling behavior of the implementations vs
> each other.

I did experiment with that initially. I found the graphs became dense and
unreadable. I need to spend more time studying Tufti to present all these
data points in a single figure. The data is all included, so please, by all
means have a crack at it. Maybe you'll see something I didn't.

 As you've got it, the results seem somewhat obvious ("as the hit ratio
> gets worse, it gets slower").

Yes, that's true. Of interest in this particular experiment was the
relative performance of different caches under identical workloads.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message