hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <st...@duboce.net>
Subject Re: Improving hbase read performance
Date Wed, 18 Feb 2009 17:23:35 GMT
On Wed, Feb 18, 2009 at 8:39 AM, shourabh rawat <mirage1987@gmail.com>wrote:

> Sorry to bug u again

Its no trouble. Lets figure it out.

> well i pasted my code a few posts back...Is it the same as wat u r sayin...

Pardon, I only just saw it.

Looks like you are setting up a thread pool of 50 threads and then each time
the thread runs, it gets one value only?  Each thread makes its own HTable

Set up a pool of 10 threads and have them each get 1000 values and see what
your numbers are like?  Or run ten processes each fetching 1000 values.

I say 10 because with 50, the single Connection is probably a bottleneck.  I
also say 1000 so the cost of thread setup is amorticized..

0.20.0 hopefully will be out in a month or two.  There is still a bunch of
work to be done.

"You could also run multiple clients each to their own process so each
process got its own Connection instance."

Didn't get wat u mean by  this...
> Well is it possible to get multiple connection instances. Isn't that
> the property of the HTables and with same name they alwyas have the
> same connection instances.
> Could you give some sample code which cld help me on this "multiple
> connection instances"

I was suggesting that you invoke your client program ten times,
concurrently: e.g for i in "1..10"; do java YOURPROGRAM &; done (something
like that).  You'd need to let it run longer so cost of jvm setup would wash


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message