hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ramkrishna vasudevan <ramkrishna.s.vasude...@gmail.com>
Subject Re: Did major compaction really happen?
Date Fri, 20 Dec 2013 05:07:12 GMT
Sometimes if the list of files selected for minor compaction is equal to
the total store files present at that time, such minor compactions are
promoted to work as major compaction.  So this is what should have happened
in your system.
This is not a bug but a behaviour.

Regards
Ram


On Fri, Dec 20, 2013 at 10:19 AM, branky <branky34@gmail.com> wrote:

> In order to turn off major compaction, I set MAJOR_COMPACTION_PERIOD to 0.
> However, I checked region server log, saw major compaction “happened"
>  quite
> often:
>
> 2013-12-17 01:44:45,085 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed major compaction of 4 file(s) in cf1 of
> my_table,75,1387269666255.a6b0df058bf97f2f0efe1ed75f445a2c. into
> b23af74c66d8477b963750feed19ef23, size=90.2m; total size for store is 90.2m
> 2013-12-17 01:45:33,762 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed major compaction of 4 file(s) in cf1 of
> my_table,75,1387269666255.a6b0df058bf97f2f0efe1ed75f445a2c. into
> 5a9bcff84c9e4a96bc40c31d4ad80956, size=157.5m; total size for store is
> 179.7m
> 2013-12-17 01:46:29,629 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed major compaction of 4 file(s) in cf1 of
> my_table,75,1387269666255.a6b0df058bf97f2f0efe1ed75f445a2c. into
> 8000d825d4fa4a9bb10d2aef2820e5c5, size=226.2m; total size for store is
> 248.4m
> 2013-12-17 01:47:26,814 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed major compaction of 4 file(s) in cf1 of
> my_table,75,1387269666255.a6b0df058bf97f2f0efe1ed75f445a2c. into
> 724dd3f1c4ae4e7fb3925f70f878c724, size=294.8m; total size for store is
> 317.1m
> 2013-12-17 01:56:51,058 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed major compaction of 4 file(s) in cf1 of
> my_table,75,1387269666255.a6b0df058bf97f2f0efe1ed75f445a2c. into
> efc0238399bc4e40bf86cf76e5b55ed9, size=546.9m; total size for store is
> 613.8m
> 2013-12-17 02:05:49,823 INFO org.apache.hadoop.hbase.regionserver.Store:
> Completed major compaction of 5 file(s) in cf1 of
> my_table,75,1387269666255.a6b0df058bf97f2f0efe1ed75f445a2c. into
> d296aa41ea6c4ddb8e552d8cb89b03dc, size=1.0g; total size for store is 1.1g
>
> But then when I checked the detail log further, I were totally confused:
> 2013-12-17 01:55:48,930 DEBUG org.apache.hadoop.hbase.regionserver.Store:
> a6b0df058bf97f2f0efe1ed75f445a2c - cf1: Initiating minorcompaction
> 2013-12-17 01:55:48,930 INFO org.apache.hadoop.hbase.regionserver.HRegion:
> Starting compaction on cf1 in region
> my_table,75,1387269666255.a6b0df058bf97f2f0efe1ed75f445a2c.
> 2013-12-17 01:55:48,930 DEBUG
> org.apache.hadoop.hbase.regionserver.CompactSplitThread:
> Small Compaction requested:
> regionName=my_table,75,1387269666255.a6b0df058bf97f2f0efe1ed75f445a2c.,
> storeName=cf1, fileCount=4, fileSize=543.9m (294.8m, 204.7m, 22.2m, 22.3m),
> priority=3, time=9701090312106379; Because: Thread-15;
> compaction_queue=(0:0),
> split_queue=0
> 2013-12-17 01:55:48,930 INFO
> org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 4
> file(s) in
> cf1 of my_table,75,1387269666255.a6b0df058bf97f2f0efe1ed75f445a2c. into
> tmpdir=h
> dfs://
> namenode.mycompany.com/hbase/my_table/a6b0df058bf97f2f0efe1ed75f445a2c/.tm
> p, seqid=2261767047, totalSize=543.9m 2013-12-17 01:56:51,058 INFO
> org.apache.hadoop.hbase.regionserver.Store: Completed major compaction of 4
> file(s) in cf1 of
> my_table,75,1387269666255.a6b0df058bf97f2f0efe1ed75f445a2c.
> into efc0238399bc4e40bf86cf76e5b55ed9, size=546.9m; total size for store is
> 613.8m
> 2013-12-17 01:56:51,058 INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactionRequest:
> completed
> compaction:
> regionName=my_table,75,1387269666255.a6b0df058bf97f2f0efe1ed75f445a2c.,
> storeName=cf1, fileCount=4, fileSize=543.9m, priority=3,
> time=9701090312106379;
> duration=1mins, 2sec
>
>
> It seems a minor compaction was initiated, but then turned to be
> identified as
> major compaction when it’s finished. I am not sure whether this is a bug
> or not,
> can anyone explain this situation? Thank you!
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message