Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
In-Reply-To: 
 <CAGpTDNeCVxBR8WdSxPtATfngHercyniPgjA=hj3A9PQ5Rx_Bkw@mail.gmail.com>
Subject: Re: HTablePool for Reading Performance
To: user@hbase.apache.org
Message-ID: 
 <OF29E69A9D.25BE4E17-ON482578D5.00027B0C-482578D5.0002FA9F@tsmc.com>
From: hmchiud@tsmc.com
Date: Fri, 22 Jul 2011 08:32:32 +0800
MIME-Version: 1.0
Content-type: text/plain; charset=Big5


Hi,
I've loaded 30 oracle tables into HBase successfully. All these columns are
in a single columnFamily.
My java program is trying to random read a few data from the first table to
the last.
Like Bill of Material, my user will pass one or a few products to my
application which will
return each product's materials to user. For that, I try to create thread
in a machine for fetching data from Hbase quickly.
Each thread shares one configure.
Following is my thread reading result:

   Thread Count      duration(second)
       1                                     69
       2                                     97,98
       3                                     106,117,119
      10
193,193,198,203,203,207,209,212,216,217

That would be better if the duration can keep at around 70~80 while 10
threads are running.
Any suggestions? Thank you very much

>Our cluster with 20 machines(1U server, 4 cores, 12G RAM)
>Hadoop             20 machines
>Zookeeper          3
>RegionServer   10
>Hadoop version hadoop-0.20.2-cdh3u0
>HBase version   hbase-0.90.1-cdh3u0


Fleming Chiu(������)
Ext: 707-2260
Be Veg, Go Green, Save the Planet!


             jdcryans@gmail.com                                                                                                           
                                                                                                                                          
                                                                                                                                       To 
             Sent by:                          user@hbase.apache.org                                                                      
             jdcryans@gmail.com                                                                                                        cc 
                                                                                                                                          
                                                                                                                                  Subject 
             2011/07/22 �W�� 05:28             Re: HTablePool for Reading Performance                                                     
                                                                                                                                          
                                                                                                                                          
                 Please respond to                                                                                                        
               user@hbase.apache.org                                                                                                      
                                                                                                                                          
                                                                                                                                          
Inline.

J-D

> I am trying to start multiple threads in a client to read data from
Hbase.
> These threads seem to share only one connection, which resulted in
> increasing latency.

Data on that?

> HTablePool can create multiple connections, right?

Multiple HTable, but they all share one connection to each RS and the
queries/answers are multiplexed.

> I am using two ways to scan HBASE data.
> One is scanning by Key value, the other is scanning by index
> (hbase-transactional-tableindexed IndexedTable)
> I don't know how to adopt HTablePool for my IndexedTable.

I think you just need to pass a class that implements
HTableInterfaceFactory to the constructor and that creates
IndexedTable instead of HTable.

J-D


 --------------------------------------------------------------------------- 
                                                         TSMC PROPERTY       
 This email communication (and any attachments) is proprietary information   
 for the sole use of its                                                     
 intended recipient. Any unauthorized review, use or distribution by anyone  
 other than the intended                                                     
 recipient is strictly prohibited.  If you are not the intended recipient,   
 please notify the sender by                                                 
 replying to this email, and then delete this email and any copies of it     
 immediately. Thank you.                                                     
 ---------------------------------------------------------------------------