hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Venkatesh <vramanatha...@aol.com>
Subject Re: hbase 0.90.2 - incredibly slow response
Date Fri, 22 Apr 2011 03:21:07 GMT
Thanks St. Ack..
Sorry I had to roll back to 0.20.6..as our system is down way too long..
so..i don't have log rt now..i'll try to recreate in a different machine at a later time..

yes..700 mil puts per day
cluster is 20 node (20 datanode+ region server) ..besides that 1 machine with HMaster &
1 name node, 3 zoo keeper

we do , new HTable, put, close in a multi threaded servlet (tomcat based) (HBase configuration
object is constructed in init)
same logic works great in 0.20.6..
In 0.90.2 all I changed was retrofit HTable constructor ..& it crawls

we rolled back to 0.20.6 & it works great again..obviously some major logic change in
0.90.2
that requires perhaps different coding practice for client api..If you can shed some light
that wld be helpfull

My hbase config is pretty much default except region size (using 4gig)

 


 

 

-----Original Message-----
From: Stack <stack@duboce.net>
To: user@hbase.apache.org
Sent: Wed, Apr 20, 2011 2:11 pm
Subject: Re: hbase 0.90.2 - incredibly slow response


Want to paste your configuration up in pastebin?



Is that 700million puts a day?



Remind us of your cluster size.



Paste some of a regionserver log too.  That can be informative.



St.Ack



On Wed, Apr 20, 2011 at 10:41 AM, Venkatesh <vramanathan00@aol.com> wrote:

> shell is no problems..ones/twos..i've tried mass puts from shell

> we cant handle our production load (even 1/3 of it)

> 700 mill per day is full load..same load we handled with absolutely no issues 

in 0.20.6..

> there is several pause between batch of puts as wel

>

>

>

>

>

>

>

>

>

>

> -----Original Message-----

> From: Stack <stack@duboce.net>

> To: user@hbase.apache.org

> Sent: Wed, Apr 20, 2011 1:30 pm

> Subject: Re: hbase 0.90.2 - incredibly slow response

>

>

> On Tue, Apr 19, 2011 at 11:58 AM, Venkatesh <vramanathan00@aol.com> wrote:

>

>>

>

>>  I was hoping that too..

>

>> I don't have scripts to generate # requests from shell..I will try that..

>

>>

>

>

>

> Did you try it?

>

>

>

> Above you seem to say that a simple put of < 100 bytes takes 2 seconds

>

> where in 0.20.6 it took < 10 milliseconds.  A put from shell of 100

>

> bytes is easy enough to do.

>

>

>

> hbase> put 'YOUR_TABLE', 'SOME_ROW', 'SOME_COLUMN', 'SOME_STRING_OF_100_BYTES'

>

>

>

> The shell will print out rough numbers on how long it takes to do the

>

> put (from ruby).

>

>

>

>

>

>> I did n't pre-create regions in 0.20.6 & it handled fine the same load..

>

>> I'll try performance in 0.90.2 by precreating regions..

>

>>

>

>> Would sharing a single HBaseConfiguration object for all threads hurt

>

> performance?

>

>>

>

>

>

> I'd doubt that this is the issue.  It should help usually.

>

>

>

> St.Ack

>

>

>

>


 

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message