hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hmch...@tsmc.com
Subject Re: HTablePool for Reading Performance
Date Fri, 22 Jul 2011 00:32:32 GMT

I've loaded 30 oracle tables into HBase successfully. All these columns are
in a single columnFamily.
My java program is trying to random read a few data from the first table to
the last.
Like Bill of Material, my user will pass one or a few products to my
application which will
return each product's materials to user. For that, I try to create thread
in a machine for fetching data from Hbase quickly.
Each thread shares one configure.
Following is my thread reading result:

   Thread Count      duration(second)
       1                                     69
       2                                     97,98
       3                                     106,117,119

That would be better if the duration can keep at around 70~80 while 10
threads are running.
Any suggestions? Thank you very much

>Our cluster with 20 machines(1U server, 4 cores, 12G RAM)
>Hadoop             20 machines
>Zookeeper          3
>RegionServer   10
>Hadoop version hadoop-0.20.2-cdh3u0
>HBase version   hbase-0.90.1-cdh3u0

Fleming Chiu(邱宏明)
Ext: 707-2260
Be Veg, Go Green, Save the Planet!

             Sent by:                          user@hbase.apache.org                     
             2011/07/22 上午 05:28             Re: HTablePool for Reading Performance  
                 Please respond to                                                       



> I am trying to start multiple threads in a client to read data from
> These threads seem to share only one connection, which resulted in
> increasing latency.

Data on that?

> HTablePool can create multiple connections, right?

Multiple HTable, but they all share one connection to each RS and the
queries/answers are multiplexed.

> I am using two ways to scan HBASE data.
> One is scanning by Key value, the other is scanning by index
> (hbase-transactional-tableindexed IndexedTable)
> I don't know how to adopt HTablePool for my IndexedTable.

I think you just need to pass a class that implements
HTableInterfaceFactory to the constructor and that creates
IndexedTable instead of HTable.


                                                         TSMC PROPERTY       
 This email communication (and any attachments) is proprietary information   
 for the sole use of its                                                     
 intended recipient. Any unauthorized review, use or distribution by anyone  
 other than the intended                                                     
 recipient is strictly prohibited.  If you are not the intended recipient,   
 please notify the sender by                                                 
 replying to this email, and then delete this email and any copies of it     
 immediately. Thank you.                                                     

View raw message