hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nicolas Spiegelberg <nspiegelb...@fb.com>
Subject Re: manual trigger of major compaction does not seems to work on 0.92
Date Fri, 24 Feb 2012 16:36:16 GMT
In 0.92, we can handle multiple compactions but we wait on major compact
if minor compactions are ongoing for the same store.  It should be
enqueued and work after the minor compaction finishes, it's just not
immediately enqueued.  See HBASE-5330 for some discussion on this.  There
should be a log message in the regionserver to verify the the
MAJOR_COMPACT request was received.  The existing contract on major
compaction requests are that they are asynchronous & the request has
in-memory persistence.

On 2/24/12 4:32 AM, "Mikael Sitruk" <mikael.sitruk@gmail.com> wrote:

>I triggered several time major_compaction between 22:30 and the time i
>wrote the original email 00:58
>From the log of one of the severs between those time no major compaction
>was running a major_compaction ended at 22:23 on the same server but this
>is because the number of file was high (after a load phase) and not
>manually triggered.
>On the system major_compaction are disabled.
>
>From the logs:
>1. 22:23:42 --> 22:23:48 - single major compaction runs
>2. 22:23:48 --> 23:44:27 - minor compaction runs
>3. 23:44:27 --> 23:44:32 - single major compaction runs
>5. 23:44:32 --> 01:13:49 - minor compaction are run
>5. 01:13:49 the cycle of major compaction began to run (heavily working
>only from 01:40:).
>-------------------------------
>*below a grep of compaction (ends, with major and minor to illustrate the
>above)*
>
>2012-02-23 22:23:48,272 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 1 file(s) in PI of
>TC,00000000000012921999,1329399349811.8dcc952ff57e87c6d82840e84eb870a6.
>into 7400a662b5924d3baf3ac520b1eb7ad9, size=49.0m; total size for store is
>49.0m
>2012-02-23 22:25:38,477 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000010451999,1329399349799.38739d7acb5f10198540bcfa0d4b5aef.
>into 77c43127e38846f78e7ada302d399732, size=540.8m; total size for store
>is
>2.1g
>2012-02-23 22:27:41,930 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000003951999,1329399349764.f55cfda6067ad1425ba68d20d82c945b.
>into f9c31ac78b3d4feb848b1e8d78975e34, size=532.0m; total size for store
>is
>2.1g
>2012-02-23 22:29:43,260 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000010971999,1329399349802.15720106584fe32f58287b6174970f31.
>into af2472f30a9f4c558323a3268f98183c, size=517.9m; total size for store
>is
>2.6g
>2012-02-23 22:30:19,476 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000010061999,1329399349797.27a2f37d5d6ac48c860cca5251be5731.
>into 3ac255c439ab4c518f4a938d542c9220, size=335.4m; total size for store
>is
>2.1g
>2012-02-23 22:31:15,051 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000008501999,1329399349789.fa0ac64d8ac9050f4acf2b0deb6e2dc7.
>into b8c733a27d02468a82f8a2c4d84db4ff, size=523.0m; total size for store
>is
>2.1g
>2012-02-23 22:32:33,430 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000004991999,1329399349770.46a2a16d484faacfcda36bc52bb218ba.
>into 5344911081424de69299128412d41573, size=538.0m; total size for store
>is
>2.2g
>2012-02-23 22:33:28,882 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000004081999,1329399349765.a5ca0a6a0a5561af5b6964e458615493.
>into d15c1e5b089d46beb2c4d01c796367a7, size=520.7m; total size for store
>is
>2.1g
>2012-02-23 22:34:46,714 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 3 file(s) in EVENT of
>TC,00000000000007591999,1329399349784.711b032969e1ae6cce8e16bb9b
>...
>..
>..
>2012-02-23 23:44:32,215 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 1 file(s) in PI of
>TC,00000000000000051999,1329399349734.e7ba4a35ee8524fbf4e6f9f9ba845206.
>into 2c701bd613314198889d8b3d79b82992, size=40.8m; total size for store is
>40.8m
>2012-02-23 23:45:00,838 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000005251999,1329399349771.50df7d847b5e36be9982b47eb3234ad4.
>into 701d49ca010a41169ca495917b4ce3d4, size=1.3g; total size for store is
>2.1g
>2012-02-23 23:47:12,191 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000002261999,1329399349751.ffb8f2f3d9d5a23aee280359793557ae.
>into d2b8336a36b1451d9a7d38c6bf3e79ee, size=1.2g; total size for store is
>2.1g
>2012-02-23 23:49:28,440 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000011881999,1329399349806.81f2bb0de10974b617521b62b9b85f09.
>into 19208d50fe8543a682e3351c60801eb9, size=1.2g; total size for store is
>2.1g
>2012-02-23 23:52:12,523 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000000051999,1329399349734.e7ba4a35ee8524fbf4e6f9f9ba845206.
>into 639ffb3d8c434363aac0d007b6f78ce9, size=1.5g; total size for store is
>2.1g
>2012-02-23 23:56:14,058 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000005901999,1329399349775.48aa14a1feea4793ec68bfe076570257.
>into 3e2f293a33f14612b8fa1ad570799feb, size=1.6g; total size for store is
>2.1g
>2012-02-23 23:59:08,211 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000010451999,1329399349799.38739d7acb5f10198540bcfa0d4b5aef.
>into dd7f92dd786d4e3ea9cb733cb3a6d2f8, size=1.6g; total size for store is
>2.1g
>...
>...
>...
>...
>2012-02-24 01:17:15,611 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 6 file(s) in EVENT of
>TC,00000000000012401999,1329399349808.03a3d34ac2c0c7d09619
>8032a151120f. into eff7b5bda7bf4653a9f229c1b40e0e4f, size=1.3g; total size
>for store is 2.1g
>2012-02-24 01:17:51,057 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000000441999,1329399349737.02354040d2650db1d59de0e01b
>0edd1d. into ac1d5e5f1ab746229bad223f56534054, size=329.8m; total size for
>store is 2.2g
>2012-02-24 01:20:40,454 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 5 file(s) in EVENT of
>TC,00000000000012921999,1329399349811.8dcc952ff57e87c6d828
>40e84eb870a6. into ae82b7832eed4bafb9e64ed45574831c, size=978.9m; total
>size for store is 2.5g
>2012-02-24 01:21:13,635 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000007981999,1329399349786.7e26a6440421112a0a15ed78ac
>33b168. into 94aa6f0328834b0896a05cda6a6a9a0f, size=309.1m; total size for
>store is 2.1g
>2012-02-24 01:22:50,578 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000002391999,1329399349752.05a0d235efa8135618628e5124
>dc9507. into 0ca4a3b8221c42f992649431e7182a6d, size=329.6m; total size for
>store is 2.2g
>2012-02-24 01:25:03,055 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 5 file(s) in EVENT of
>TC,00000000000001221999,1329399349743.56d16361d17dce747fb0
>23ab15ded16b. into 47e9f68efb8a475dbce7db2c08b46f9c, size=1.0g; total size
>for store is 2.5g
>2012-02-24 01:25:38,876 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000001091999,1329399349742.5907e62062b4edc83033b4fda0
>fc105f. into d5148c6b4dbc4d3c8339f151b987e51a, size=326.4m; total size for
>store is 2.2g
>2012-02-24 01:26:10,242 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000005381999,1329399349772.15d2f2fb1ad338ab3b5e57077d
>874601. into bacf547a2b4f4627b630a18b88ec8ec6, size=306.5m; total size for
>store is 2.1g
>2012-02-24 01:27:47,690 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000010581999,1329399349800.8144106ed55b1571a720e46569
>af8ea6. into 827d75a8b3144ee3a7e53dae5fdecf3a, size=296.5m; total size for
>store is 2.1g
>2012-02-24 01:28:23,907 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000006681999,1329399349779.9e0dc98e734715e66702419dc2
>1e23ed. into 4a93423a056645d7a6f869f66c76fd0e, size=341.4m; total size for
>store is 2.1g
>2012-02-24 01:29:11,192 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000012661999,1329399349810.76715703e8bf5f3f960564834f
>d9417d. into 3ef0ae62a13a4ed98cd772eeb1e55fd0, size=438.1m; total size for
>store is 2.1g
>2012-02-24 01:29:49,930 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000011101999,1329399349802.a63a43695fdcc0be243cfe3ea6
>263981. into 2a542d20b2ed49b5964777c97f5f2a5b, size=355.2m; total size for
>store is 2.1g
>2012-02-24 01:33:08,747 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 5 file(s) in EVENT of
>TC,00000000000008761999,1329399349791.7c0b5c9e5c98e9e39ae7
>a3afcfaad4d9. into 93fb6ab4de7a4672aceed0e23cc9effc, size=1.2g; total size
>for store is 2.1g
>2012-02-24 01:35:21,194 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 5 file(s) in EVENT of
>TC,00000000000006811999,1329399349780.6051d2a9230460194d41
>f6249d087f6f. into 3adba15b5ae04bd1a18a18afeb7cf107, size=1.2g; total size
>for store is 2.1g
>2012-02-24 01:37:08,338 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000011621999,1329399349805.4d2bb3d00b0fa089163d6a0491dedadb.
>into 26ac704d826a495080e93f07b3c6e24d, size=395.6m; total size for store
>is
>2.6g
>2012-02-24 01:37:51,454 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000012011999,1329399349807.99d15c956a3897276d0fdeaa33611417.
>into 28aac65938bf493187f84e7019ca126d, size=393.8m; total size for store
>is
>2.1g
>2012-02-24 01:38:33,930 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000004861999,1329399349769.dce64ca59ca897a34aa6416de4f32dea.
>into 5dda42912eb64701b20639065a7612cb, size=393.6m; total size for store
>is
>2.1g
>2012-02-24 01:39:23,057 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed compaction of 4 file(s) in EVENT of
>TC,00000000000010841999,1329399349801.c7f801818067718327858c10b0f3ded2.
>into 39ce298723bc4f66a5bfe1f0729143d4, size=468.0m; total size for store
>is
>2.2g
>2012-02-24 01:40:16,369 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 4 file(s) in EVENT of
>TC,00000000000001611999,1329399349746.eb6f8b45363b6d3f254d4b6d3dab3617.
>into 7821cb65510840afaff12c1e008b28c4, size=491.1m; total size for store
>is
>2.1g
>2012-02-24 01:42:18,657 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 4 file(s) in EVENT of
>TC,00000000000002651999,1329399349754.5c75673b41370540e3c758e937181651.
>into a7a12651ff3d414bb722945157686c11, size=505.1m; total size for store
>is
>2.1g
>2012-02-24 01:43:10,404 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 4 file(s) in EVENT of
>TC,00000000000004211999,1329399349766.041d7f3c86c4d5657c6eeded7584fad8.
>into 7b8d8075a4d04873a50e517431935e03, size=482.2m; total size for store
>is
>2.1g
>2012-02-24 01:44:03,388 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 4 file(s) in EVENT of
>TC,00000000000011751999,1329399349805.21ac7408f58fa1b022b6a38b2f197915.
>into f2906f916c16442a89161fdf9551bc0c, size=507.9m; total size for store
>is
>2.1g
>2012-02-24 01:44:59,108 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 4 file(s) in EVENT of
>TC,00000000000010061999,1329399349797.27a2f37d5d6ac48c860cca5251be5731.
>into 60defcf94fce43488fffc619c509a9d3, size=524.2m; total size for store
>is
>2.1g
>2012-02-24 01:46:17,683 INFO org.apache.hadoop.hbase.regionserver.Store:
>Completed major compaction of 4 file(s) in EVENT of
>TC,00000000000011491999,1329399349804.c613c26eff42730acd1f
>..
>..
>..
>..
>
>
>Mikael.S
>
>
>On Fri, Feb 24, 2012 at 2:05 AM, Jean-Daniel Cryans
><jdcryans@apache.org>wrote:
>
>> Can you show us some logs that exhibit this behavior?
>>
>> Thx,
>>
>> J-D
>>
>> On Thu, Feb 23, 2012 at 2:58 PM, Mikael Sitruk <mikael.sitruk@gmail.com>
>> wrote:
>> > Hi
>> >
>> > Before filling a jira, I  would like to check this with you.
>> > It seems that the major compaction behavior changed between 0.90.x and
>> 0.92
>> > (not taking about all the nice stuff you added there...)
>> >
>> > Effectively when i trigger a major compaction (via hbase shell or ui)
>> when
>> > other compactions (minor) are running (because i had a lot of
>>storefiles
>> > and a big compactionQueueSize), the major compaction is not executed.
>> While
>> > in 0.90.X major_compact command was executed immediately after any
>> current
>> > compaction.
>> >
>> >
>> > Mikael.S
>>
>
>
>
>-- 
>Mikael.S


Mime
View raw message