Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of saint.ack@gmail.com designates
 209.85.214.169 as permitted sender)
MIME-Version: 1.0
Sender: saint.ack@gmail.com
In-Reply-To: 
 <CAPud8TpwQoHeyPQ3AK5MZYh_Cn25wbAYDYeT20CyFm_5Rsw=vA@mail.gmail.com>
References: 
 <CAPud8TpwQoHeyPQ3AK5MZYh_Cn25wbAYDYeT20CyFm_5Rsw=vA@mail.gmail.com>
Date: Wed, 18 Jan 2012 09:34:07 -0800
Message-ID: 
 <CADcMMgG3qVwbbnZMSZ9tArauD_bmLDwT95=21H1OzzCAb1bYGQ@mail.gmail.com>
Subject: Re: Table region got stuck, doesn't move/assign
From: Stack <stack@duboce.net>
To: user@hbase.apache.org
Content-Type: text/plain; charset=ISO-8859-1

On Mon, Jan 16, 2012 at 3:45 PM, Dmitriy Lyubimov <dlieu.7@gmail.com> wrote:
> i have a table which seems to get stuck in a state where it can't be
> queried, moved or split/compacted.
>

How many regions in this table?  One only?

> The logs don't have any error statements. Our admin tried hbck to no avail .
>

What did your admin see?


> We stopped the region server, table did not get reassigned. (all other
> did). when bround in UI, this table just showed "region server
> offline". (??? shouldn't get reassigned as others did?)
>

Yes.  It should.

> Brining region server online loaded it with other regions, but not
> that table. master apparently still thinks it is on that node (data6)
> and so all requests are failing with region not serving message.
>


So, there is something 'wrong' w/ that table.   Can you track it in
master log and see what happens when master tries assign it?  Maybe
its failing to open?

> assign/move/ unassign commands have no effect (move fails, but
> assing/unassign seems to be quiet with no apparent effect).
>
> Another weirdness: it's the only table that is showing up under
> hbase/table in zk and its region is listed under /hbase/unassigned.
>


Maybe its stuck in transition?  You should see messages in master log
if this the case.

> Where can i read about meaning and transitions of zookeeper nodes under /hbase ?
>

I don't think this documented in the reference guide (its a little too
much detail for most I'd say).  Best place to look is probably source
code.  See here for an entrance into the wonderful world of
master/regionserver state transitions:
http://hbase.apache.org/xref/org/apache/hadoop/hbase/executor/EventHandler.html#93

St.Ack