Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 76FE4952A for ; Mon, 12 Dec 2011 22:40:03 +0000 (UTC) Received: (qmail 49540 invoked by uid 500); 12 Dec 2011 22:40:03 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 49512 invoked by uid 500); 12 Dec 2011 22:40:03 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 49504 invoked by uid 99); 12 Dec 2011 22:40:03 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Dec 2011 22:40:03 +0000 X-ASF-Spam-Status: No, hits=-2001.5 required=5.0 tests=ALL_TRUSTED,RP_MATCHES_RCVD X-Spam-Check-By: apache.org Received: from [140.211.11.116] (HELO hel.zones.apache.org) (140.211.11.116) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 12 Dec 2011 22:39:53 +0000 Received: from hel.zones.apache.org (hel.zones.apache.org [140.211.11.116]) by hel.zones.apache.org (Postfix) with ESMTP id BE84D10EB48 for ; Mon, 12 Dec 2011 22:39:30 +0000 (UTC) Date: Mon, 12 Dec 2011 22:39:30 +0000 (UTC) From: "Hudson (Commented) (JIRA)" To: issues@hbase.apache.org Message-ID: <2048702700.3149.1323729570781.JavaMail.tomcat@hel.zones.apache.org> In-Reply-To: <795968937.62595.1323607120014.JavaMail.tomcat@hel.zones.apache.org> Subject: [jira] [Commented] (HBASE-5008) The clusters can't provide services because Region can't flush. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 X-Virus-Checked: Checked by ClamAV on apache.org [ https://issues.apache.org/jira/browse/HBASE-5008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13167897#comment-13167897 ] Hudson commented on HBASE-5008: ------------------------------- Integrated in HBase-TRUNK #2540 (See [https://builds.apache.org/job/HBase-TRUNK/2540/]) HBASE-5008 The clusters can't provide services because Region can't flush. stack : Files : * /hbase/trunk/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java > The clusters can't provide services because Region can't flush. > ---------------------------------------------------------------- > > Key: HBASE-5008 > URL: https://issues.apache.org/jira/browse/HBASE-5008 > Project: HBase > Issue Type: Bug > Components: regionserver > Reporter: gaojinchao > Assignee: gaojinchao > Priority: Blocker > Fix For: 0.92.0, 0.90.6 > > Attachments: HBASE-5008_Branch90.patch > > > Hbase version 0.90.4 + patches > My analysis is as follows: > //Started splitting region b24d8ccb852ff742f2a27d01b7f5853e and closed region. > 2011-12-10 17:32:48,653 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: Starting split of region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. > 2011-12-10 17:32:49,759 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Closing Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: disabling compactions & flushes > 2011-12-10 17:32:49,759 INFO org.apache.hadoop.hbase.regionserver.HRegion: Running close preflush of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. > //Processed a flush request and skipped , But flushRequested had set to true > 2011-12-10 17:33:06,963 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Started memstore flush for Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e., current region memstore size 12.6m > 2011-12-10 17:33:17,277 DEBUG org.apache.hadoop.hbase.regionserver.HRegion: Skipping flush on Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. because closing > //split region b24d8ccb852ff742f2a27d01b7f5853 failed and rolled back, flushRequested flag was true, So all handle was blocked > 2011-12-10 17:34:01,293 INFO org.apache.hadoop.hbase.regionserver.SplitTransaction: Cleaned up old failed split transaction detritus: hdfs://193.195.18.121:9000/hbase/Htable_UFDR_004/b24d8ccb852ff742f2a27d01b7f5853e/splits > 2011-12-10 17:34:01,294 INFO org.apache.hadoop.hbase.regionserver.HRegion: Onlined Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.; next sequenceid=15494173 > 2011-12-10 17:34:01,295 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: Successful rollback of failed split of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. > 2011-12-10 17:43:10,147 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 19 on 20020' on region > Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is >= than blocking 384.0m size > // All handles had been blocked. The clusters could not provide services > 2011-12-10 17:34:01,295 INFO org.apache.hadoop.hbase.regionserver.CompactSplitThread: Successful rollback of failed split of Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e. > 2011-12-10 17:43:10,147 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 19 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is >= than blocking 384.0m size > 2011-12-10 17:43:10,192 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 34 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is >= than blocking 384.0m size > 2011-12-10 17:43:10,193 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 51 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is >= than blocking 384.0m size > 2011-12-10 17:43:10,196 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 85 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is >= than blocking 384.0m size > 2011-12-10 17:43:10,199 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 88 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is >= than blocking 384.0m size > 2011-12-10 17:43:10,202 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 44 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is >= than blocking 384.0m size > 2011-12-10 17:43:11,663 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 2 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is >= than blocking 384.0m size > 2011-12-10 17:43:11,665 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 10 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is >= than blocking 384.0m size > 2011-12-10 17:43:11,670 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 75 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is >= than blocking 384.0m size > 2011-12-10 17:43:11,671 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 98 on 20020' on region Htable_UFDR_004,09781,1323508582833.b24d8ccb852ff742f2a27d01b7f5853e.: memstore size 384.0m is >= than blocking 384.0m size > 2011-12-10 17:43:11,680 INFO org.apache.hadoop.hbase.regionserver.HRegion: Blocking updates for 'IPC Server handler 11 on 20020' on region -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira