hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: How to avoid update blocking
Date Sat, 11 Feb 2017 21:08:38 GMT
Which hbase release are you using ?

You have given estimate on write load. How about read load ?
Do you observe blocking / slow down on write path ?

Consider increasing memstore, decreasing block cache if read load is light.
I assume the 4 nodes with 125 GB Ram have HDD (instead of SSD). If you have
SSD, you can use SSD for WAL - see HBASE-12848

bq. its on weekend when load is low.

This should have low impact on write load and is good practice.

Cheers

On Mon, Feb 6, 2017 at 9:15 PM, Manjeet Singh <manjeet.chandhok@gmail.com>
wrote:

> How to avoid read/write update blocking in hbase and on what condition
> Hbase block read/write?
>
>
>
> what should the frequency of Major compaction ( I have daily data insert on
> average is 18 GB and 48 GB max, its with SNAPPY and FAST_DIFF)
>
>  a. should it be on daily basis?
>
>  b. should it be in weekends?
>
>  c. does Major compaction on daily basis will avoid blocking?
>
>
> What is the calculation? Right now its on weekend when load is low.
>
> we have :
> total 7 (3+4) data nodes
> 3 80GB Ram 1.8TB HDD
> 4 125 GB Ram HDD 6 TB
>
> below are few important configurations:
>
> hbase.regionserver.global.memstore.lowerLimit = .38
> hbase.regionserver.global.memstore.upperLimit = .4
> hbase.hregion.memstore.block.multiplier=2
> hbase.hregion.majorcompaction=0 ( scheduled on weekend)
> hbase.hregion.memstore.flush.size 128 MB
> hbase.hregion.preclose.flush.size 5 MB
> hbase.hstore.compactionThreshold 3
>
> --
> luv all
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message