hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: Performance at large number of regions/node
Date Sat, 29 May 2010 17:53:50 GMT
On Fri, May 28, 2010 at 4:11 PM, Jacob Isaac <jacob@ebrary.com> wrote:
> Here is the summary of the runs
>
> puts (~4-5k per row)
> regionsize #rows       Total time (ms)
> 1G 82282053*2      301943742
> 512M 82287593*2      313119378
> 256M 82246314*2      433200105
>

So about 0.3ms per 5k write (presuming 100M writes?)?


> gets ((~4-5k per row)
> regionsize #rows      Total time (ms)
> 1G 82427685        90116726
> 512M 82421943        94878466
> 256M 82395487        108160178
>

How many reads are you doing?  Are they going on concurrent with the writing?


> Note : for the 256m run the hbase.hregion.memstore.flush.size=64m
> and for the other two runs the hbase.hregion.memstore.flush.size=96m
>

I wonder what is going on in a typical regionserver log during these
runs?  Do you see lots of blocking going on (we'll block if memory is
full or compaction has been overrun -- with the 96M flushing you might
be generating lots of store files provoking lots of compacting -- as
per the issue you cite earlier by jgray.  If you look in master UI, is
there a steady state of requests across all regionservers?  Or do they
fall to zero alot?  Would be good to check regionserver log at these
times).

Are the servers loaded at all?

Thanks Jacob,
St.Ack

Mime
View raw message