Return-Path: 
 <user-return-41635-apmail-hbase-user-archive=hbase.apache.org@hbase.apache.org>
X-Original-To: apmail-hbase-user-archive@www.apache.org
Delivered-To: apmail-hbase-user-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 2B8BD100A2
	for <apmail-hbase-user-archive@www.apache.org>;
 Fri, 14 Feb 2014 23:59:42 +0000 (UTC)
Received: (qmail 94407 invoked by uid 500); 14 Feb 2014 23:59:37 -0000
Delivered-To: apmail-hbase-user-archive@hbase.apache.org
Received: (qmail 94336 invoked by uid 500); 14 Feb 2014 23:59:36 -0000
Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:user-help@hbase.apache.org>
List-Unsubscribe: <mailto:user-unsubscribe@hbase.apache.org>
List-Post: <mailto:user@hbase.apache.org>
List-Id: <user.hbase.apache.org>
Reply-To: user@hbase.apache.org
Delivered-To: mailing list user@hbase.apache.org
Received: (qmail 94327 invoked by uid 99); 14 Feb 2014 23:59:36 -0000
Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Feb 2014 23:59:36 +0000
X-ASF-Spam-Status: No, hits=1.5 required=5.0
	tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (nike.apache.org: domain of yuzhihong@gmail.com designates
 209.85.160.177 as permitted sender)
Received: from [209.85.160.177] (HELO mail-yk0-f177.google.com)
 (209.85.160.177)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Feb 2014 23:59:29 +0000
Received: by mail-yk0-f177.google.com with SMTP id q200so25003361ykb.8
        for <user@hbase.apache.org>; Fri, 14 Feb 2014 15:59:08 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
        d=gmail.com; s=20120113;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        bh=PKGnteEoa20wxZ+Vm1L+R+iKLDe0f0lymxezYNW10DM=;
        b=v9zwVDRuAgFC4cerMzcOY+fWjQkC65AsMVhT7Nv2BxvVcOyo8YjohEuuqrga58ALou
         Rju35e/rirpOCTOELPNmfMkRatNZVhpwr7Gl0AkaD5tE8sJ9TIMmVxF5GT23LbB86tst
         Z0C+YFMkOJAawakXlas+Ink4nocV8jXUHXPYP2F57H3It/3ZaA8wzeP51aCZl49X7kYD
         wjdUTUJSecoB++ZRtDYMJd/YoRGJUpaJV+yWhiUc0707ITbvAJYMdcZ08VtrnDtZ6StO
         4Od4jE56yYIiHsq5y86XXyKts0KycOJsdjNgdlS0oelTE/mvINAJk/s1DaO7A+wANVtQ
         yOzQ==
MIME-Version: 1.0
X-Received: by 10.236.174.37 with SMTP id w25mr5278506yhl.36.1392422348752;
 Fri, 14 Feb 2014 15:59:08 -0800 (PST)
Received: by 10.170.213.65 with HTTP; Fri, 14 Feb 2014 15:59:08 -0800 (PST)
In-Reply-To: 
 <CALCVZZuMUfLDxgz87F3jjBF90Z8+vH-4xdXZnv7NsAjEDhOh7g@mail.gmail.com>
References: 
 <CALCVZZuMUfLDxgz87F3jjBF90Z8+vH-4xdXZnv7NsAjEDhOh7g@mail.gmail.com>
Date: Fri, 14 Feb 2014 15:59:08 -0800
Message-ID: 
 <CALte62z5uAWF4DbO6Sa32WuWh4iF1AdJ1m85GP-nPtvstDmo6A@mail.gmail.com>
Subject: Re: uneven region distribution
From: Ted Yu <yuzhihong@gmail.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Content-Type: multipart/alternative; boundary=20cf305b0ab4798c7904f26697ca
X-Virus-Checked: Checked by ClamAV on apache.org

--20cf305b0ab4798c7904f26697ca
Content-Type: text/plain; charset=ISO-8859-1

bq.  it does not change the status of the assignments.

Can you check / pastebin master log to see what caused the balancing to
stop ?

bq. attributing the region server crash to the disproportionately high
number of regions on that server?

Checking region server log on server5 should give us more clue.

bq. 0.92.4

please consider upgrading :-)


On Fri, Feb 14, 2014 at 3:52 PM, Rohit Kelkar <rohitkelkar@gmail.com> wrote:

> I am using hbase version 0.92.4 on a 5 node cluster. I am seeing that a
> particular region server often crashes. A status 'simple' on hbase shell
> gives the following stats
>
>
> HBase Shell; enter 'help<RETURN>' for list of supported commands. Type
> "exit<RETURN>" to leave the HBase Shell Version 0.94.2, r1395367, Sun Oct 7
> 19:11:01 UTC 2012
> status 'simple' 4 live servers
> server7:60020 1392017875910 requestsPerSecond=0, numberOfOnlineRegions=419,
> usedHeapMB=3315, maxHeapMB=6127
> server4:60020 1392300859332 requestsPerSecond=843,
> numberOfOnlineRegions=379, usedHeapMB=2070, maxHeapMB=6127
> server3:60020 1391583646998 requestsPerSecond=429,
> numberOfOnlineRegions=653, usedHeapMB=3198, maxHeapMB=6127
> server6:60020 1391583647588 requestsPerSecond=0, numberOfOnlineRegions=966,
> usedHeapMB=2975, maxHeapMB=6127 1 dead servers
> server5,60020,1392108515637 Aggregate load: 1272, regions: 2417
>
> The dead region server has 2417 regions as opposed to 419, 379, 653, 966
> regions on other servers. Am I right in attributing the region server crash
> to the disproportionately high number of regions on that server?
>
> If I invoke the balancer on hbase shell using the "balancer" command it
> returns true. But it does not change the status of the assignments.
>
> - R
>

--20cf305b0ab4798c7904f26697ca--