hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Serega Sheypak <serega.shey...@gmail.com>
Subject Spikes when writing data to HBase
Date Tue, 11 Aug 2015 10:22:33 GMT
Hi, we are using version 1.0.0+cdh5.4.4+160
We have heavy write load, ~ 10K per econd
We have 10 nodes 7 disks each. I read some perf notes, they state that
HBase can handle 1K per second writes per node without any problems.

I see some spikes on "writers". Write operation timing "jumps" from 40-50ms
to 200-500ms Probably I hit memstore limit. RegionServer starts to flush
memstore and stop to accept updates.

I have several questions:
1. Does 4/(8 in hyperthreading) CPU + 7HDD node could absorb 1K writes per
2. What is the right way to fight with blocked writes?
2.1. What I did:
hbase.hregion.memstore.flush.size to 256M to produce larger HFiles when
flushing memstore
base.hregion.memstore.block.multiplier to 4, since I have only one
intensive-write table. Let it grow
hbase.regionserver.optionallogflushinterval to 10s, i CAN loose some data,
NP here. The idea that I reduce I/O pressure on disks.
Not sure if I can correctly play with these parameters.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message