Return-Path: X-Original-To: apmail-hbase-dev-archive@www.apache.org Delivered-To: apmail-hbase-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9C83AC9DD for ; Mon, 3 Jun 2013 06:27:23 +0000 (UTC) Received: (qmail 43659 invoked by uid 500); 3 Jun 2013 06:27:22 -0000 Delivered-To: apmail-hbase-dev-archive@hbase.apache.org Received: (qmail 43396 invoked by uid 500); 3 Jun 2013 06:27:21 -0000 Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hbase.apache.org Delivered-To: mailing list dev@hbase.apache.org Received: (qmail 43379 invoked by uid 99); 3 Jun 2013 06:27:20 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 03 Jun 2013 06:27:20 +0000 Date: Mon, 3 Jun 2013 06:27:19 +0000 (UTC) From: "Calvin Qu (JIRA)" To: dev@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Created] (HBASE-8676) hbase.hstore.compaction.max.size parameter has no effect in minor compaction MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 Calvin Qu created HBASE-8676: -------------------------------- Summary: hbase.hstore.compaction.max.size parameter has no effect in minor compaction Key: HBASE-8676 URL: https://issues.apache.org/jira/browse/HBASE-8676 Project: HBase Issue Type: Bug Components: regionserver Affects Versions: 0.94.7 Reporter: Calvin Qu In hbase-site.xml,I set(10G) hbase.hstore.compaction.max.size 10737418240 but in regionserver's log,a minor compaction deals with 10 Hfiles,include a 220G HFile larger than hbase.hstore.compaction.max.size=10G. 2013-06-03 10:27:51,147 DEBUG org.apache.hadoop.hbase.regionserver.Store: 8b6a4d4aae3099730b353183cf754ec4 - t: Initiating minorcompaction 2013-06-03 10:27:51,149 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on t in region transdb,1651763699103243149223370692144515807,1369372490168.8b6a4d4aae3099730b353183cf754ec4. 2013-06-03 10:27:51,150 DEBUG org.apache.hadoop.hbase.regionserver.CompactSplitThread: Large Compaction requested: regionName=transdb,1651763699103243149223370692144515807,1369372490168.8b6a4d4aae3099730b353183cf754ec4., storeName=t, fileCount=10, fileSize=234.4g (106.0m, 2.0g, 2.1g, 10.7m, 1.7g, 2.1g, 1.9g, 2.0g, 932.8m, 221.5g), priority=-110, time=2671513455305347; Because: Opening Region; compaction_queue=(0:0), split_queue=0 2013-06-03 10:27:51,151 INFO org.apache.hadoop.hbase.regionserver.Store: Starting compaction of 10 file(s) in t of transdb,1651763699103243149223370692144515807,1369372490168.8b6a4d4aae3099730b353183cf754ec4. into tmpdir=hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/.tmp, seqid=0, totalSize=234.4g 2013-06-03 10:27:51,152 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/0a5fb3caf6be490ba739b77fcaa4b274.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/0a5fb3caf6be490ba739b77fcaa4b274-top, keycount=7948080, bloomtype=NONE, size=106.0m, encoding=NONE 2013-06-03 10:27:51,153 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/6bd849c93e3a45d4a4c8a5ab691f9b22.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/6bd849c93e3a45d4a4c8a5ab691f9b22-top, keycount=153911753, bloomtype=NONE, size=2.0g, encoding=NONE 2013-06-03 10:27:51,153 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/191ef6d1076341e1b7fb5c775d0f9fdb.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/191ef6d1076341e1b7fb5c775d0f9fdb-top, keycount=159372725, bloomtype=NONE, size=2.1g, encoding=NONE 2013-06-03 10:27:51,154 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/d907a7dca7d8466f843f126429f05665.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/d907a7dca7d8466f843f126429f05665-top, keycount=808800, bloomtype=NONE, size=10.7m, encoding=NONE 2013-06-03 10:27:51,154 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/35399e73e3764d0b88251c7285cf5406.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/35399e73e3764d0b88251c7285cf5406-top, keycount=133146091, bloomtype=NONE, size=1.7g, encoding=NONE 2013-06-03 10:27:51,154 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/9d5583ffde334bc79a692d400af7bae0.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/9d5583ffde334bc79a692d400af7bae0-top, keycount=159372708, bloomtype=NONE, size=2.1g, encoding=NONE 2013-06-03 10:27:51,155 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/506254eac1bb455283f578e71af23153.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/506254eac1bb455283f578e71af23153-top, keycount=152578349, bloomtype=NONE, size=1.9g, encoding=NONE 2013-06-03 10:27:51,155 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/72db7a39a2694bf58782e95992b012ae.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/72db7a39a2694bf58782e95992b012ae-top, keycount=159369079, bloomtype=NONE, size=2.0g, encoding=NONE 2013-06-03 10:27:51,155 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/128f5cebbf074129b7c59de3d436b57d.c7ce1266ef1f8696ede82f08fceb28ff-hdfs://namenode01:9000/hbase/transdb/c7ce1266ef1f8696ede82f08fceb28ff/t/128f5cebbf074129b7c59de3d436b57d-top, keycount=72366990, bloomtype=NONE, size=932.8m, encoding=NONE 2013-06-03 10:27:51,155 DEBUG org.apache.hadoop.hbase.regionserver.Compactor: Compacting hdfs://namenode01:9000/hbase/transdb/8b6a4d4aae3099730b353183cf754ec4/t/df7ac534daa645589e5e2a6fe1a1a871, keycount=17218754243, bloomtype=NONE, size=221.5g, encoding=NONE -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira