Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2F03617690 for ; Tue, 24 Feb 2015 16:31:39 +0000 (UTC) Received: (qmail 5011 invoked by uid 500); 24 Feb 2015 16:31:37 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 4942 invoked by uid 500); 24 Feb 2015 16:31:37 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 4930 invoked by uid 99); 24 Feb 2015 16:31:37 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Feb 2015 16:31:37 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW X-Spam-Check-By: apache.org Received-SPF: error (athena.apache.org: local policy) Received: from [209.85.220.169] (HELO mail-vc0-f169.google.com) (209.85.220.169) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 24 Feb 2015 16:31:32 +0000 Received: by mail-vc0-f169.google.com with SMTP id kv19so10324440vcb.0 for ; Tue, 24 Feb 2015 08:29:21 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=3AVVbw0qhTHMUdI4quJJjClpVblNTQdE8pAlH04zjEY=; b=VjCHaIaV0TZ1aUf/LZzzfT4A3Ns9ahLoJrEmh8z0L6Ij97KHB4yoOyoevmsqpVPTIw qQDJXEDbwr5mFtTCqoYzbYPh405EAJJ46zut/LV5MgC50cLLTkYxbxoOwu3oYufRdOCt ug7cwB/N1V1UT9nNqDX5tL4VjvX3yx71J3LJq05Iv+jWVPuSW12HoPUlIFfivgzkKngC FPLx31Z8EOs3Jj2iQaRP8ZGvKpbTVZ0gourkqJbaBQtnQ4iLIoxAz1hpxHibxnbEMJPM GntTiQZ7RtZCUbbEqhWZ3JKE9ceWfNJIYcGFtGMFczjNukO9aTcNXwAxhL0PxRAFVosn hEyA== X-Gm-Message-State: ALoCoQlmdL6mPFDNvu+QW3WXlxIwGg1Sm+9lpnERijJx+LQ8FkIzk7/x2vUoC1HFoeFRZmJqo0kE X-Received: by 10.52.110.134 with SMTP id ia6mr16491846vdb.94.1424795361134; Tue, 24 Feb 2015 08:29:21 -0800 (PST) MIME-Version: 1.0 Received: by 10.52.146.139 with HTTP; Tue, 24 Feb 2015 08:28:59 -0800 (PST) In-Reply-To: <547B12E3-3A93-49C7-AAD7-F6F95219D28E@digitalenvoy.net> References: <547B12E3-3A93-49C7-AAD7-F6F95219D28E@digitalenvoy.net> From: Jean-Marc Spaggiari Date: Tue, 24 Feb 2015 11:28:59 -0500 Message-ID: Subject: Re: periodicFlusher get stuck To: user Content-Type: multipart/alternative; boundary=bcaec54861b660d601050fd8051e X-Virus-Checked: Checked by ClamAV on apache.org --bcaec54861b660d601050fd8051e Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Interesting... Can you share you hbase-site.xml? Have you setup hbase.regionserver.optionalcacheflushinterval? Can you hadoop fs -ls -R this region folder? 2015-02-24 11:15 GMT-05:00 Brian Jeltema : > I=E2=80=99m seeing occasional HBase log output similar to the output show= n below. > It appears there is a request to flush a region, repeated every 10 > seconds, that apparently is never being performed. It=E2=80=99s causing M= R jobs to > timeout because they cannot write to this region. Is this a known problem= ? > > hbase version 0.98.0.2.1.2.1-471-hadoop2 > hadoop version 2.4.0.2.1.2.1-471 > > > 2015-02-23 14:51:47,612 INFO [regionserver60020.periodicFlusher] > regionserver.HRegionServer: regionserver60020.periodicFlusher requesting > flush for region > Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after= a > delay of 13758 > 2015-02-23 14:51:57,611 INFO [regionserver60020.periodicFlusher] > regionserver.HRegionServer: regionserver60020.periodicFlusher requesting > flush for region > Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after= a > delay of 18080 > 2015-02-23 14:52:07,611 INFO [regionserver60020.periodicFlusher] > regionserver.HRegionServer: regionserver60020.periodicFlusher requesting > flush for region > Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after= a > delay of 17701 > 2015-02-23 14:52:17,612 INFO [regionserver60020.periodicFlusher] > regionserver.HRegionServer: regionserver60020.periodicFlusher requesting > flush for region > Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after= a > delay of 19090 > 2015-02-23 14:52:27,616 INFO [regionserver60020.periodicFlusher] > regionserver.HRegionServer: regionserver60020.periodicFlusher requesting > flush for region > Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after= a > delay of 4042 > 2015-02-23 14:52:37,615 INFO [regionserver60020.periodicFlusher] > regionserver.HRegionServer: regionserver60020.periodicFlusher requesting > flush for region > Host,\x00_m\xB8\x06,1415044750009.6ec50faa43a312cd6465d991e5984ec6. after= a > delay of 12968 > 2015-02-23 18:12:03,307 INFO [regionserver60020.periodicFlusher] > regionserver.HRegionServer: regionserver60020.periodicFlusher requesting > flush for region > Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after= a > delay of 10482 > 2015-02-23 18:12:13,308 INFO [regionserver60020.periodicFlusher] > regionserver.HRegionServer: regionserver60020.periodicFlusher requesting > flush for region > Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after= a > delay of 14829 > 2015-02-23 19:15:13,330 INFO [regionserver60020.periodicFlusher] > regionserver.HRegionServer: regionserver60020.periodicFlusher requesting > flush for region > Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after= a > delay of 22888 > 2015-02-23 19:15:23,329 INFO [regionserver60020.periodicFlusher] > regionserver.HRegionServer: regionserver60020.periodicFlusher requesting > flush for region > Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after= a > delay of 21081 > 2015-02-23 19:15:33,329 INFO [regionserver60020.periodicFlusher] > regionserver.HRegionServer: regionserver60020.periodicFlusher requesting > flush for region > Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after= a > delay of 6387 > 2015-02-23 20:50:23,368 INFO [regionserver60020.periodicFlusher] > regionserver.HRegionServer: regionserver60020.periodicFlusher requesting > flush for region > Host,\x00_m\xB8\x06,1424724136146.48d4d3fa0e02a97a8a1d9b85d5cf0162. after= a > delay of 8828 > > --bcaec54861b660d601050fd8051e--