Mailing-List: contact hadoop-dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hadoop-dev@lucene.apache.org
Received-SPF: neutral (herse.apache.org: local policy)
Message-ID: <46800337.50109@troove.net>
Date: Mon, 25 Jun 2007 11:02:31 -0700
From: James Kennedy <james.kennedy@troove.net>
User-Agent: Thunderbird 1.5.0.2 (X11/20060420)
MIME-Version: 1.0
To: hadoop-dev@lucene.apache.org
Subject: HBase HClient - Can we make it multi-table?
References: <e8ce6bc30706201625x2a35340eu30d9f9742024ca02@mail.gmail.com>
	 <467AB5F5.4050605@apache.org>
 <e8ce6bc30706211403k66565872ke433ecef6223fa79@mail.gmail.com>
In-Reply-To: <e8ce6bc30706211403k66565872ke433ecef6223fa79@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

I have an app that needs to access multiple HBase tables concurrently.  
The current HClient can only have one table open at a time even though 
it caches region servers of multiple tables as they are looked up.

This means that my application layer must open multiple HClients, one 
per table, perhaps caching those HClients in a pool to reuse them (and 
their cached table data) as appropriate.

or

Shall I write an HClient patch that makes the HClient  multi-table 
thread-safe?

Essentially this would mean changing the interface to add table name 
parameters to methods like put(), commit(), startUpdate() etc, maintain 
a list of opened tables (as opposed to the single tableServers field), 
and ensure that it is thread-safe.  I would also keep for each table a  
<row, lockid and currentServer and currentRegion> map for startUpdate() 
calls so that the the different rows of the same table could be updated 
concurrently.  This would also make it possible to  check for the rare 
case of clientid = rand.nextLong() collisions, if that's deemed necessary.

Alternatively, perhaps what we need is something like an 
HClientManager/Factory which manages table info but  gives out 
single-table HClients on request to be used for one-off transactions.  
This keeps the HClient simpler while keeping the cached regionInfo shared. 

Thoughts?