hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Performance at large number of regions/node
Date Fri, 28 May 2010 20:20:02 GMT
On Fri, May 28, 2010 at 1:13 PM, Jacob Isaac <jacob@ebrary.com> wrote:
> Hi J-D
>
> hbase.regionserver.maxlogs was 256 although
> hbase.regionserver.hlog.blocksize was the default.
>
> Did not use compression. And autoflush is default (true)

You should, and if you are uploading in big batches then disable
autoflush then make sure you commit.

>
> Each of the 20 node is running custom server program that's reading and
> writing to HBase
> Max of 6 write threads per node and 1 thread reading
> Also wanted to point out that in the current tests we are writing to two
> tables and reading from only one

That's a very important information! Currently reading is slower than
writing in HBase (unless you are scanning lots of rows at a time,
using scanner caching) so this might easily be your bottleneck right?
Do you benchmark all the steps by any chance? Can you show at the end
of your (for a lack of better word) "testing" the average/total time
it took to read/write?

J-D

Mime
View raw message