hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dean <hikeonp...@gmail.com>
Subject Re: Off-heap block cache fails in 0.94.6
Date Thu, 23 Jan 2014 18:01:00 GMT
Hi Ramkrishna,

I had taken the experimental label to mean "use at your own risk" when 
the feature was released in 0.92.  After some judicious testing in the 
spirit of "use at your own risk", we found the feature worked well for 
our use case.  In 0.94, we had expected off-heap cache to be the same as 
0.92, or perhaps even with a few small improvements. It came as a bit of 
a surprise that the feature went from experimental to plain broken.

Perhaps my interpretation of "experimental" was off base to begin with - 
in retrospect it may have been presumptuous to assume that an 
"experimental" feature would only become more mature and eventually 
graduate to a full-fledged feature.  I can see how such a feature might 
go the other way, quietly going from experimental to deprecated to broken.

We'd love a viable off-heap cache solution - we're currently scrambling 
a bit for options in production since we're not able to roll back to 
0.92.1.  The HBASE-7404 solution that Ted pointed out looks awesome, but 
we're not in a position to move to 0.96 to take advantage of it.  At 
this point, I'm fishing for options.  We run a mix of random reads and 
scans.  We disable block cache for scans to improve random read 
performance.  With a chunk of RAM allocated to off-heap cache, this 
seemed to work well.  Now that we're dominated by filesystem cache, 
scans with block cache disabled still trash the cache, significantly 
hurting our random read performance.  We've got short-circuit reads and 
Hbase checksums enabled in an attempt to gain back some of the lost 
perf, but they are more of a band-aid than a fix for root cause.  Any 
suggestions are much appreciated.

Cheers,

Dean

Mime
View raw message