Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EA8C810A6F for ; Mon, 16 Sep 2013 12:31:56 +0000 (UTC) Received: (qmail 10286 invoked by uid 500); 16 Sep 2013 12:31:55 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 10253 invoked by uid 500); 16 Sep 2013 12:31:54 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 10244 invoked by uid 99); 16 Sep 2013 12:31:53 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 Sep 2013 12:31:53 +0000 Date: Mon, 16 Sep 2013 12:31:53 +0000 (UTC) From: "Hadoop QA (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-9467) write can be totally blocked temporarily by a write-heavy region MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-9467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13768282#comment-13768282 ] Hadoop QA commented on HBASE-9467: ---------------------------------- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12603321/HBASE-9467-trunk-v1.patch against trunk revision . {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+1 tests included{color}. The patch appears to include 3 new or modified tests. {color:green}+1 hadoop1.0{color}. The patch compiles against the hadoop 1.0 profile. {color:green}+1 hadoop2.0{color}. The patch compiles against the hadoop 2.0 profile. {color:green}+1 javadoc{color}. The javadoc tool did not generate any warning messages. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:green}+1 findbugs{color}. The patch does not introduce any new Findbugs (version 1.3.9) warnings. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:green}+1 site{color}. The mvn site goal succeeds with this patch. {color:red}-1 core tests{color}. The patch failed these unit tests: org.apache.hadoop.hbase.coprocessor.TestMasterObserver org.apache.hadoop.hbase.coprocessor.TestAggregateProtocol org.apache.hadoop.hbase.mapred.TestTableInputFormat org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed org.apache.hadoop.hbase.mapreduce.TestRowCounter org.apache.hadoop.hbase.io.encoding.TestChangingEncoding org.apache.hadoop.hbase.client.TestHTableUtil org.apache.hadoop.hbase.mapreduce.TestImportTsv org.apache.hadoop.hbase.coprocessor.TestOpenTableInCoprocessor org.apache.hadoop.hbase.coprocessor.TestClassLoading org.apache.hadoop.hbase.thrift.TestThriftServer org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster org.apache.hadoop.hbase.trace.TestHTraceHooks org.apache.hadoop.hbase.mapreduce.TestCopyTable org.apache.hadoop.hbase.mapreduce.TestImportExport org.apache.hadoop.hbase.client.TestHTablePool$TestHTableThreadLocalPool org.apache.hadoop.hbase.util.TestMergeTool org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFiles org.apache.hadoop.hbase.security.access.TestTablePermissions org.apache.hadoop.hbase.snapshot.TestExportSnapshot org.apache.hadoop.hbase.TestZooKeeper org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithRemove org.apache.hadoop.hbase.client.TestCloneSnapshotFromClient org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher org.apache.hadoop.hbase.client.TestClientTimeouts org.apache.hadoop.hbase.client.TestSnapshotCloneIndependence org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan2 org.apache.hadoop.hbase.client.TestFromClientSideNoCodec org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildOverlap org.apache.hadoop.hbase.client.TestMultiParallel org.apache.hadoop.hbase.mapred.TestTableMapReduce org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildBase org.apache.hadoop.hbase.security.access.TestAccessControlFilter org.apache.hadoop.hbase.coprocessor.TestRegionServerCoprocessorExceptionWithAbort org.apache.hadoop.hbase.thrift.TestThriftServerCmdLine org.apache.hadoop.hbase.client.TestTimestampsFilter org.apache.hadoop.hbase.util.TestRegionSplitter org.apache.hadoop.hbase.catalog.TestMetaMigrationConvertingToPB org.apache.hadoop.hbase.client.TestMetaScanner org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache org.apache.hadoop.hbase.coprocessor.TestRegionObserverBypass org.apache.hadoop.hbase.client.TestAdmin org.apache.hadoop.hbase.client.TestMultipleTimestamps org.apache.hadoop.hbase.master.TestAssignmentManagerOnCluster org.apache.hadoop.hbase.master.handler.TestCreateTableHandler org.apache.hadoop.hbase.master.TestMasterMetricsWrapper org.apache.hadoop.hbase.master.TestMasterRestartAfterDisablingTable org.apache.hadoop.hbase.TestAcidGuarantees org.apache.hadoop.hbase.master.TestRollingRestart org.apache.hadoop.hbase.TestFullLogReconstruction org.apache.hadoop.hbase.coprocessor.TestRegionObserverScannerOpenHook org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient org.apache.hadoop.hbase.coprocessor.TestBigDecimalColumnInterpreter org.apache.hadoop.hbase.mapreduce.TestTableMapReduce org.apache.hadoop.hbase.mapreduce.TestWALPlayer org.apache.hadoop.hbase.client.TestScannersFromClientSide org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithRemove org.apache.hadoop.hbase.mapreduce.TestCellCounter org.apache.hadoop.hbase.TestIOFencing org.apache.hadoop.hbase.regionserver.TestCompaction org.apache.hadoop.hbase.mapreduce.TestHLogRecordReader org.apache.hadoop.hbase.master.TestMasterTransitions org.apache.hadoop.hbase.client.TestScannerTimeout org.apache.hadoop.hbase.client.TestClientScannerRPCTimeout org.apache.hadoop.hbase.util.TestMergeTable org.apache.hadoop.hbase.regionserver.TestServerCustomProtocol org.apache.hadoop.hbase.client.TestShell org.apache.hadoop.hbase.master.TestRestartCluster org.apache.hadoop.hbase.mapreduce.TestMultithreadedTableMapper org.apache.hadoop.hbase.mapreduce.TestSecureLoadIncrementalHFilesSplitRecovery org.apache.hadoop.hbase.coprocessor.TestMasterCoprocessorExceptionWithAbort org.apache.hadoop.hbase.util.TestMiniClusterLoadParallel org.apache.hadoop.hbase.client.TestSnapshotMetadata org.apache.hadoop.hbase.client.TestHTablePool$TestHTableReusablePool org.apache.hadoop.hbase.TestDrainingServer org.apache.hadoop.hbase.util.TestMiniClusterLoadSequential org.apache.hadoop.hbase.master.TestMasterFileSystem org.apache.hadoop.hbase.master.TestZKBasedOpenCloseRegion org.apache.hadoop.hbase.zookeeper.TestZooKeeperACL org.apache.hadoop.hbase.util.TestCoprocessorScanPolicy org.apache.hadoop.hbase.master.TestOpenedRegionHandler org.apache.hadoop.hbase.io.TestFileLink org.apache.hadoop.hbase.master.TestMasterMetrics org.apache.hadoop.hbase.client.TestHTableMultiplexer org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFiles org.apache.hadoop.hbase.master.TestMasterFailover org.apache.hadoop.hbase.mapreduce.TestLoadIncrementalHFilesSplitRecovery org.apache.hadoop.hbase.backup.TestHFileArchiving org.apache.hadoop.hbase.master.TestTableLockManager org.apache.hadoop.hbase.master.handler.TestTableDescriptorModification org.apache.hadoop.hbase.coprocessor.TestRowProcessorEndpoint org.apache.hadoop.hbase.mapreduce.TestHRegionPartitioner org.apache.hadoop.hbase.client.TestHCM org.apache.hadoop.hbase.master.TestMasterShutdown org.apache.hadoop.hbase.client.TestSnapshotFromClient org.apache.hadoop.hbase.coprocessor.TestWALObserver org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient org.apache.hadoop.hbase.client.TestFromClientSide org.apache.hadoop.hbase.util.TestMiniClusterLoadEncoded org.apache.hadoop.hbase.master.TestRegionPlacement org.apache.hadoop.hbase.client.TestFromClientSide3 org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor org.apache.hadoop.hbase.mapreduce.TestTableInputFormatScan1 org.apache.hadoop.hbase.security.access.TestAccessController org.apache.hadoop.hbase.TestLocalHBaseCluster org.apache.hadoop.hbase.catalog.TestMetaReaderEditor org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient org.apache.hadoop.hbase.coprocessor.TestCoprocessorEndpoint org.apache.hadoop.hbase.master.TestDistributedLogSplitting org.apache.hadoop.hbase.util.TestFSUtils org.apache.hadoop.hbase.util.hbck.TestOfflineMetaRebuildHole org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat org.apache.hadoop.hbase.coprocessor.TestRegionObserverInterface org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster org.apache.hadoop.hbase.master.cleaner.TestHFileCleaner org.apache.hadoop.hbase.master.TestMaster org.apache.hadoop.hbase.io.encoding.TestLoadAndSwitchEncodeOnDisk org.apache.hadoop.hbase.regionserver.wal.TestLogRolling org.apache.hadoop.hbase.util.TestHBaseFsck Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/7240//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/7240//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/7240//console This message is automatically generated. > write can be totally blocked temporarily by a write-heavy region > ---------------------------------------------------------------- > > Key: HBASE-9467 > URL: https://issues.apache.org/jira/browse/HBASE-9467 > Project: HBase > Issue Type: Improvement > Reporter: Feng Honghua > Assignee: Feng Honghua > Attachments: HBASE-9467-trunk-v0.patch, HBASE-9467-trunk-v1.patch > > > Write to a region can be blocked temporarily if the memstore of that region reaches the threshold(hbase.hregion.memstore.block.multiplier * hbase.hregion.flush.size) until the memstore of that region is flushed. > For a write-heavy region, if its write requests saturates all the handler threads of that RS when write blocking for that region occurs, requests of other regions/tables to that RS also can't be served due to no available handler threads...until the pending writes of that write-heavy region are served after the flush is done. Hence during this time period, from the RS perspective it can't serve any request from any table/region just due to a single write-heavy region. > This sounds not very reasonable, right? Maybe write requests from a region can only be served by a sub-set of the handler threads, and then write blocking of any single region can't lead to the scenario mentioned above? > Comment? -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira