hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ramkrishna vasudevan <ramkrishna.s.vasude...@gmail.com>
Subject Re: Off-heap block cache fails in 0.94.6
Date Fri, 24 Jan 2014 04:46:04 GMT
Hi Dean

I mentioned those JIRAs to highlight the fact that the bug was open and it
was not fixed.  I think better we fix this in 0.94 also, so that users
using this feature do not run into surprises.  And we are sorry that the
feature that worked well for you in 0.92, did not work out in 0.94.  It
should be resolved IMO.

You are going for a patch from your side?  If so, that would be great.

Regards
Ram


On Thu, Jan 23, 2014 at 11:31 PM, Dean <hikeonpast@gmail.com> wrote:

> Hi Ramkrishna,
>
> I had taken the experimental label to mean "use at your own risk" when the
> feature was released in 0.92.  After some judicious testing in the spirit
> of "use at your own risk", we found the feature worked well for our use
> case.  In 0.94, we had expected off-heap cache to be the same as 0.92, or
> perhaps even with a few small improvements. It came as a bit of a surprise
> that the feature went from experimental to plain broken.
>
> Perhaps my interpretation of "experimental" was off base to begin with -
> in retrospect it may have been presumptuous to assume that an
> "experimental" feature would only become more mature and eventually
> graduate to a full-fledged feature.  I can see how such a feature might go
> the other way, quietly going from experimental to deprecated to broken.
>
> We'd love a viable off-heap cache solution - we're currently scrambling a
> bit for options in production since we're not able to roll back to 0.92.1.
>  The HBASE-7404 solution that Ted pointed out looks awesome, but we're not
> in a position to move to 0.96 to take advantage of it.  At this point, I'm
> fishing for options.  We run a mix of random reads and scans.  We disable
> block cache for scans to improve random read performance.  With a chunk of
> RAM allocated to off-heap cache, this seemed to work well.  Now that we're
> dominated by filesystem cache, scans with block cache disabled still trash
> the cache, significantly hurting our random read performance.  We've got
> short-circuit reads and Hbase checksums enabled in an attempt to gain back
> some of the lost perf, but they are more of a band-aid than a fix for root
> cause.  Any suggestions are much appreciated.
>
> Cheers,
>
> Dean
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message