hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saad Mufti <saad.mu...@gmail.com>
Subject HBase Write Performance Under Auto-Split
Date Wed, 27 Apr 2016 15:27:34 GMT
Hi,

Does anyone have experience with HBase write performance under auto-split
conditions? Out keyspace is randomized so all regions roughly start
auto-splitting around the same time, although early on when we had the 1024
regions we started with, they all decided to do so within an hour or so and
now that we're up to 6000 regions the process seems to be spread over 12
hours or more as they slowly reach their size thresholds.

During this time, our writes, for which we use a shared BufferedMutator
suffer as writes time out and the underlying AsyncProcess thread pool seems
to fill up. Which means callers to our service see their response times
shoot up as they spend time trying to drain the buffer and submit mutations
to the thread pool. So overall system time suffers and we can't keep up
with our input load.

Are there any guidelines on the size of the BufferedMutator to use? We are
even considering running performance tests without the BufferedMutator to
see if it is buying us anything. Currently we have it sized pretty large at
around 50 MB but maybe having it too big is not a good idea.

Any help/advice would be most appreciated.

Thanks.

----
Saad

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message