hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sven Herzky <sven.her...@mail.rakuten.com>
Subject Reducing compaction queue more efficiently
Date Thu, 27 Mar 2014 11:36:20 GMT
Hey all,

I've put some data (~ 2.5 TB) into an HBase table on a small cluster (8 
dn/rs + 1 master, max region size 10 GB, having ~ 350 regions) and 
collected something around 1500 entries in my compaction queue (~ 200 
per regionserver). Unfortunately, HBase now is slow when pushing new 
data into it and it just compacts with 0.02 per second (getting also 
"WARN org.apache.hadoop.ipc.HBaseServer: (operationTooSlow)" in log files).

If I want to trigger the compaction manually, there's nothing happening:

hbase(main):001:0> compact 'images'
0 row(s) in 1.3690 seconds

hbase(main):002:0> major_compact 'images'
0 row(s) in 2.3670 seconds

I'm currently using cdh4 with HBase 0.94.

Do somebody have a clue how to speed up the process or why the manual 
compaction doesn't work as "expected" (by me)?

Thanks in advance,

View raw message