hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Re: Rethinking the HBase client
Date Wed, 16 Nov 2011 22:41:58 GMT
Thanks Stack.

o HCM.getConnection(Configuration) is what we would use. The internal list is fine, I don't
think that needs to change (at least not for our purposes). Looking at the code it seems the
client code could also just create a new HConnectionImplementation.

o I think an ES per HCI would be good enough. Could be passed in, too.

o Didn't think creating HTables from HCI, would fly :)  Just a crazy idea I had. The flow
would be (1) get a Connection to your cluster (2) when you need to access table, ask the connection
for a proxy (i.e. an HTable instance).

(What's wrong with HCI? :))

OK, I'll work out a simple patch that allows HTable to be optionally created with an externally
managed ExecutorService and HConnection instance. That would definitely be good enough for

-- Lars

From: Stack <stack@duboce.net>
To: dev@hbase.apache.org; lars hofhansl <lhofhansl@yahoo.com>
Sent: Wednesday, November 16, 2011 2:13 PM
Subject: Re: Rethinking the HBase client

On Wed, Nov 16, 2011 at 1:53 PM, lars hofhansl <lhofhansl@yahoo.com> wrote:
> The HConnection code itself seems clean, so ideally what we want to do is managing HConnections.
These are what identify and connect us to a cluster, and should drive everything.

Currently you can't get your hands directly on a particular
HConnection instance.  You have to do
HCM.getConnection(Configuration).  Wouldn't you want to change this
too -- i.e. not have HCM keep up an internal list?  Or at least have
the option to by-pass HCM accounting?

> HConnectionImplementation could have the thread pool on it that is shared by the various
HTables, and HTables could be created by a method on HConnection.

(Can we give this class a better name than HCI?  Smile)

You want to ask HCI to create an ExecutorService for batch operations?
   Would you want to pass it one so one ES for all HCIs in a hosting

Having HConnection create HTables seems wrong.

HTable adds the retrying Callable stuff in it.

> So... An alternate (stop-gap) approach is to add a new constructor to HTable that allows
for an optional HConnection and a ThreadPool to be provided.

When you say ThreadPool, you mean ExecutorService?   The pool instance
in HTable?

This seems fine to me.

> Long term it seems a new client is needed (maybe based on asynchbase, with an additional
synchronous layer), but that is a different story.

Sounds good to me.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message