hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: turn off size-based Major Compaction?
Date Thu, 06 Mar 2014 18:54:01 GMT
Demai:
You can find description of compaction-related parameters in:

http://hbase.apache.org/book.html#compaction

Cheers


On Thu, Mar 6, 2014 at 10:44 AM, lars hofhansl <larsh@apache.org> wrote:

> The problem is not major compactions, but compactions that compact a lot
> of files.
>
>
> HBase will always compact in the background (minor compactions).
> If a minor compaction needs to compact all files anyway it is promoted to
> a major compaction.
> There is no performance penalty to that, just that if all files are
> compacted anyway might as well get rid of the delete markers in the process.
>
>
> So rather than avoiding major compactions, what you want to avoid are
> compactions that touch a lot of (or all) files. You'd need to tweak the
> compaction setting - which is an art in itself. You could lower the
> compaction ratio, or increase the number of store files before compations
> start, pick a different compaction strategy, etc.
>
>
> -- Lars
>
>
>
> ________________________________
>  From: Demai Ni <nidmgg@gmail.com>
> To: "user@hbase.apache.org" <user@hbase.apache.org>
> Sent: Thursday, March 6, 2014 10:28 AM
> Subject: turn off size-based Major Compaction?
>
>
> hi, folks,
>
> Is there a way to turn off Major Compaction, for sure?
>
> We are using 96.0, and set hbase.hregion.majorcompaction is 0. The
> intention was to only run major compaction manually. It turns out after
> heavy load, major compaction still occur as shown in WebUI.
>
> After a bit search, I read this discussion:
>
> http://apache-hbase.679495.n3.nabble.com/Major-Compaction-Concerns-td3642142.html
> ,
> which indicated that there are three way to trigger major compaction:
> 1) user issued, (that is manually, such in hbase shell)
> 2)  timed, (that is hbase.hregion.majorcompaction)
> 3) size-based.
>
> Is there a way to prevent or configure the 3) method? I found
> hbase.hstore.compaction.max,
> but it is for minor compaction.
>
> So the question is: what configuration for the *size-based* on
> *Major*compaction?
>
> Many thanks
>
> Demai

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message