hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: asynchbase: A new entirely asynchronous / non-blocking HBase client
Date Wed, 08 Sep 2010 15:32:40 GMT
Just to say that I've been reviewing this code on and off for a while
now.  Its great stuff.  Not only is it non-blocking netty-based, at
its core it has a clone of the Twitsted Deferred [1] implementation
which looks like a sweet pattern for non-blocking network programming
but would look to be useful beyond this use case, for example
implementing a server core.


1. http://twistedmatrix.com/documents/10.1.0/api/twisted.internet.defer.Deferred.html

On Wed, Sep 8, 2010 at 2:22 AM, tsuna <tsunanet@gmail.com> wrote:
> Hello,
> I use Netty extensively, and for a long time I wanted to have a fully
> asynchronous / non-blocking and thread-safe HBase client.  So I ended
> up writing one from scratch, which I just released at:
> http://github.com/stumbleupon/asynchbase
> It's rather different from HBase's own client (HTable).  The core of
> asynchbase is made of HBaseClient (javadoc @ http://su.pr/1PJCSY), and
> you normally need only a single instance (vs. one per table with
> HTable).  This instance is entirely thread-safe (I think :D) and
> scales well in my limited loadtests (on a 4 core machine it scales
> linearly).  I wrote it originally for another project that relies
> heavily on HBase and Netty (a scalable time series database we use for
> monitoring at StumbleUpon – http://opentsdb.net), which is gonna be
> open sourced most likely during the last week of September.  In some
> write-heavy code paths I'm seeing a 3x to 4x throughput improvement
> with asynchbase.
> There's a heated debate ongoing at StumbleUpon about the license of
> the client.  Stack wants to use it in HBase itself, but the LGPLv3+ is
> unfortunately incompatible with the ASF license.  Others simply seem
> to dislike anything with the substring "GPL" in it (:D).  I haven't
> had much time to find another license yet, but I think we're going to
> switch to something like a BSD or MIT style license.  I'm going in
> vacation soon so I wanted to get some feedback from other HBase users
> instead of blocking on this stupid and annoying licensing issue.
> Your feedback or patches would be most welcome!
> PS: The source code
> (http://github.com/stumbleupon/asynchbase/blob/master/src/HBaseRpc.java)
> contains an unofficial documentation of the Hadoop and HBase RPC
> protocols as well as Hadoop's variable-length encoding for integer
> values.  I've heard that others may be interested in implementing
> native HBase clients in non-Java languages, so I thought I'd pass this
> around to save their time.
> --
> Benoit "tsuna" Sigoure
> Software Engineer @ www.StumbleUpon.com

View raw message