hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jim Kellerman <...@powerset.com>
Subject Re: HBase HClient - Can we make it multi-table?
Date Mon, 25 Jun 2007 18:43:56 GMT
My initial thought on this is to use one HClient per table and a
singleton that keeps the cached information for the root and meta
tables, since an application does not need to keep multiple copies of
them around, and, when it is necessary to resynchronize the root/meta
table caches it would only need to be done once.

Please open a Jira issue on this topic and set the component to
contrib/hbase.

Thanks.

-Jim

On Mon, 2007-06-25 at 11:02 -0700, James Kennedy wrote:
> I have an app that needs to access multiple HBase tables concurrently.  
> The current HClient can only have one table open at a time even though 
> it caches region servers of multiple tables as they are looked up.
> 
> This means that my application layer must open multiple HClients, one 
> per table, perhaps caching those HClients in a pool to reuse them (and 
> their cached table data) as appropriate.
> 
> or
> 
> Shall I write an HClient patch that makes the HClient  multi-table 
> thread-safe?
> 
> Essentially this would mean changing the interface to add table name 
> parameters to methods like put(), commit(), startUpdate() etc, maintain 
> a list of opened tables (as opposed to the single tableServers field), 
> and ensure that it is thread-safe.  I would also keep for each table a  
> <row, lockid and currentServer and currentRegion> map for startUpdate() 
> calls so that the the different rows of the same table could be updated 
> concurrently.  This would also make it possible to  check for the rare 
> case of clientid = rand.nextLong() collisions, if that's deemed necessary.
> 
> Alternatively, perhaps what we need is something like an 
> HClientManager/Factory which manages table info but  gives out 
> single-table HClients on request to be used for one-off transactions.  
> This keeps the HClient simpler while keeping the cached regionInfo shared. 
> 
> Thoughts?
-- 
Jim Kellerman, Senior Engineer; Powerset
jim@powerset.com

Mime
View raw message