hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lin Ma <lin...@gmail.com>
Subject Re: Using HBase serving to replace memcached
Date Tue, 21 Aug 2012 13:32:40 GMT
Thanks for the reply, Zahoor.

Some more comments,

1. I know very basics of Bloom filters, which is used for detect whether an
item is in a set. How to use Bloom filters in HBase to improve random read
performance? Could you show me an example? Thanks.
2. "Also more client connections is one more issue that might infest you"
-- supposing I am doing random read from a Hadoop job to access HBase, do
you mean using multiple client connections from the Hadoop job is good or
not good? Sorry I am a bit lost. :-)
3. "asynchbase will help you" -- does HBase support asynchronous API? Sorry
I cannot find it out. Appreciate if you could point me the APIs you are
referring to.

regards,
Lin

On Tue, Aug 21, 2012 at 6:55 PM, J Mohamed Zahoor <jmozah@gmail.com> wrote:

> Again. if your data is so huge that it is much larger than the available
> RAM, you might want to rethink.
> There are some configs in HBase that will help you in random read
> scenarios... like Bloom filters etc.
> Also more client connections is one more issue that might infest you...
> where connection pooling or asynchbase will help you.
>
> ./Zahoor
>
>
> On Tue, Aug 21, 2012 at 12:56 AM, Asif Ali <azifali@gmail.com> wrote:
>
> > I've used memcached heavily in such scenarios and all such data is always
> > in Memory.
> >
> > Memcached definitely is a great solution for this and scales very well.
> But
> > keep in mind - it is not consistent. Which means there are some requests
> > which will be handled incorrectly.
> >
> > Memcached is great but also look at Guava cache for similar use cases.
> >
> > Asif Ali
> >
> >
> > On Mon, Aug 20, 2012 at 9:09 AM, Lin Ma <linlma@gmail.com> wrote:
> >
> > > Thank you Drew. I like your reply, especially blocking cache nature
> > > provided by HBase. A quick question, for traditional memcached, all of
> > the
> > > items are in memory, no disk is used, correct?
> > >
> > > regards,
> > > Lin
> > >
> > > On Mon, Aug 20, 2012 at 9:26 PM, Drew Dahlke <drew.dahlke@bronto.com>
> > > wrote:
> > >
> > > > I'd say if the memcached model is working for you, stick with it.
> > > > HBase (currently) caches whole blocks. With cache blocks enabled you
> > > > can achieve 10s of thousands of reqs/sec with a pretty small cluster.
> > > > However there's a catch. Once you reach the point where your tables
> > > > are so large they can't all sit in memory at the same time you'll see
> > > > a behavior change. User traffic tends to be very random access which,
> > > > with block caching, can cause a lot of thrashing with frequent cache
> > > > evictions. We've seen this bring our cluster to it's knees.
> > > >
> > > > IMHO a better model is persist things in HBase and then cache things
> > > > with memcached just as you would with any other data store. If you're
> > > > looking for a spiffy memcached replacement I'd recommend checking out
> > > > Redis.
> > > >
> > > >
> > > > On Sat, Aug 18, 2012 at 3:12 AM, Lin Ma <linlma@gmail.com> wrote:
> > > > > Hello guys,
> > > > >
> > > > > In your experience, is it practical to use HBase directly for
> > serving?
> > > > > Saying handle directly user traffic (tens of thousands QPS scale)
> > > behind
> > > > > Apache, and replace the role of memcached? I am not sure whether
> > there
> > > > are
> > > > > any known panic to replace memcached by using HBase? One issue I
> > could
> > > > > think about is for a specific row range, only one active region
> > server
> > > > > could handle the request, but in memcached, we can setup several
> > > > memcached
> > > > > instance with duplicate content (all of them are active) to serve
> the
> > > > same
> > > > > purpose under a VIP which could achieve better performance and
> > > > scalability.
> > > > >
> > > > > Any advice or reference documents are appreciated. Thanks.
> > > > >
> > > > > regards,
> > > > > Lin
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message