hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: 0.92 and Read/writes not scaling
Date Mon, 02 Apr 2012 17:41:16 GMT
2012/4/2 Alok Singh <alok@urbanairship.com>:
> Sorry for jumping on this thread late, but, I have seen very similar
> behavior in our cluster with hadoop 0.23.2 (CDH4B2 snapshot) and hbase
> 0.23.1. We have a small, 7 node cluster (48GB/16Core/6x10Kdisk/GigE
> network) with about 500M rows/4Tb of data. The random read performance
> is excellent, but, random write throughput maxes out around 10K/sec.
> Turning off the WAL takes it up to 40-50k/sec, but, that's not
> something we will leave off in production.
> One of the settings that I experimented with was
> hbase.hregion.max.filesize. Increasing it to 10GB actually made the
> write throughput worse, so, I have set it back down to 2GB. Later this
> week, I will attempt to do another cycle of tests and hopefully have
> some thread dumps to report back with.

Thanks for writing the list Alok.  Juhani is going to come back on
this thread saying he went back to 0.20.x hadoop to get his write
performance back.  I'll let him respond.  Seems like we have an issue
w/ 0.23 hadoop WAL writes going by yours and Juhani's experience.


View raw message