hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <la...@apache.org>
Subject Re: turn off size-based Major Compaction?
Date Thu, 06 Mar 2014 18:44:35 GMT
The problem is not major compactions, but compactions that compact a lot of files.


HBase will always compact in the background (minor compactions).
If a minor compaction needs to compact all files anyway it is promoted to a major compaction.
There is no performance penalty to that, just that if all files are compacted anyway might
as well get rid of the delete markers in the process.


So rather than avoiding major compactions, what you want to avoid are compactions that touch
a lot of (or all) files. You'd need to tweak the compaction setting - which is an art in itself.
You could lower the compaction ratio, or increase the number of store files before compations
start, pick a different compaction strategy, etc.


-- Lars



________________________________
 From: Demai Ni <nidmgg@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org> 
Sent: Thursday, March 6, 2014 10:28 AM
Subject: turn off size-based Major Compaction?
 

hi, folks,

Is there a way to turn off Major Compaction, for sure?

We are using 96.0, and set hbase.hregion.majorcompaction is 0. The
intention was to only run major compaction manually. It turns out after
heavy load, major compaction still occur as shown in WebUI.

After a bit search, I read this discussion:
http://apache-hbase.679495.n3.nabble.com/Major-Compaction-Concerns-td3642142.html,
which indicated that there are three way to trigger major compaction:
1) user issued, (that is manually, such in hbase shell)
2)  timed, (that is hbase.hregion.majorcompaction)
3) size-based.

Is there a way to prevent or configure the 3) method? I found
hbase.hstore.compaction.max,
but it is for minor compaction.

So the question is: what configuration for the *size-based* on
*Major*compaction?

Many thanks

Demai
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message