hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Ruchovets <oruchov...@gmail.com>
Subject major compaction time constantly growing
Date Wed, 21 Sep 2011 18:31:05 GMT
Hi ,

 Our environment
hbase 90.2 (10 machine)
    We have 10 machine grid:
    master has 48G ram
    slaves machine has 16G ram.
    Region Server process has 4G ram
    Zookeeper process has 2G ram
    We have 4map/2reducer per machine

We write from m/r job to hbase (2 jobs a day).
   1) We running major compaction manually once a day
   2) We increases regions size to prevent automatic split
   3) We didn't use WAL (WAL is false).

We started running major compaction 3 month before on dayly basis. Major
compaction execution time from the very begining was ~1 hour. Every next day
major compaction takes more time then previous and today it took 8.5 hours.

   1) Is it correct behaviour that major compaction time constantly
growing?
   2) What is the best practice to resolve such behaviour , simply in some
time we will have major compaction which will take more then 24 hours(but we
run it on daily basis).
   3) If it is not correct behavior  how can we debug major compaction
process to verify what could cause a problem

Thanks in advance
Oleg.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message