hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frédéric Fondement <frederic.fondem...@uha.fr>
Subject Re: question on HTablePool and threads
Date Fri, 26 Aug 2011 07:29:06 GMT
Hi there,

For which versions of HBase is asyncbase working ? Can't find that on 
the website.

Does it supports external schema changes (new/deleted table/column 
family by an external thread/process)?
What about HTablePool regarding this last question ?

Is is really enough to use an HTablePool like this in multithreaded 
environment ?
e.g. (unchecked code follows)
thread1 and thread2:
while (! shouldStop) {
   HTableInterface t = pool.get("sometable");
   try {
     //no need for synchronized(t) here as both threads receive 
different instances at same time, right ?
     //works with t
   } finally {
     t.close();
   }
}

What to do when application is ended ? Is it mandatory to call 
closeTablePool ?

It would be great if HTableInterface could extend java.io.Closeable so 
that one could write
try(t) {
   //work with t
}

Sorry if questions look simple, but I'm not sure just from the Javadoc.

Cheers !

Le 26/08/2011 09:08, Ben Cuthbert a écrit :
> Hi Andy
>
> We are using the stubleupon async client. Very fast and good.
> On 26 Aug 2011, at 04:58, Zhong, Andy wrote:
>
>> Hey Michael,
>> I am also looking to the performance gain to use HTablePool instead of
>> created HTable using a singleton instance of HBaseConfiguration. If the
>> use case is for a web service to handle multi-threaded write/reading
>> from a single Hbase table, are you suggest to use HTablePool to
>> pre-created a pool of Htable instances?
>>
>> But below two comments concern me:
>> 1. Concern of restarting Hadoop/HBase cluster:
>> http://comments.gmane.org/gmane.comp.java.hadoop.hbase.user/13128
>> "The problem with the HTablePool is that it does not "ride over restart"
>> meaning that if you need to restart your cluster, HtablePool will still
>> be pointing at the old ports and not realize the cluster is back-up."
>> 2. Seems no performance gain, and even worse:
>> http://www.srikanthps.com/2011/06/hbase-benchmarking-for-multi-threaded.
>> html if no of concurrent put<10000.
>> 3. Does anyone uses glisthub: claim a fully asynchronous, non-blocking,
>> thread-safe, high-performance HBase client (not sure if anyone uses it):
>> http://github.com/stumbleupon/asynchbase
>>
>> What do you think of it, and welcome any one's advices or comments on
>> this.
>>
>> Thanks,
>> Andy Zhong
>>
>>
>> -----Original Message-----
>> From: Michael Segel [mailto:michael_segel@hotmail.com]
>> Sent: Tuesday, August 23, 2011 10:57 PM
>> To: user@hbase.apache.org
>> Subject: RE: question on HTablePool and threads
>>
>>
>> Sujee,
>>
>> You are correct in creating a separate HTable instance in each thread.
>> (HTable isn't thread safe, but since the scope is within the thread it
>> works.)
>>
>> You could use the HTablePool class, but I don't think its a better
>> solution for what you are doing.
>>
>> In your example it sounds like you're creating the connection in each
>> thread and you're using it for the life of the thread/application.  So
>> there's no real benefit in trying to create a pool of threads and then
>> request a thread from the pool.
>>
>> JMHO
>>
>> -Mike
>>
>>
>>> From: sujee@sujee.net
>>> Date: Tue, 23 Aug 2011 16:34:32 -0700
>>> Subject: question on HTablePool and threads
>>> To: hbase-user@hadoop.apache.org
>>>
>>> Hi all,
>>>
>>> Right now I have a  java client program that accesses Hbase in
>>> multiple threads for read / write.  Each thread creates its own
>>> instance of HTable of the _same_ table.
>>>
>>> I am looking into HTablePool class.  Not clear on if it is a
>>> correct/better choice for accessing the _same_ table with multiple
>> threads.
>>> Is this a valid  / thread-safe ?
>>>
>>> create HtablePool in 'main'
>>> pass HTablePool instance to threads
>>> each thread does a  'htablepool.get(table)'
>>>     read / write to table
>>>     'htablepool.put (table) '  when done (all this is done within a
>>> single JVM)
>>>
>>> thanks
>>> Sujee Maniyam
>>> http://sujee.net
>> 		 	   		
>>
>> This message, including any attachments, is the property of Sears Holdings Corporation
and/or one of its subsidiaries. It is confidential and may contain proprietary or legally
privileged information. If you are not the intended recipient, please delete it without reading
the contents. Thank you.


Mime
View raw message