hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: HMaster won't die waiting for RegionServer that is already dead
Date Fri, 21 Jan 2011 21:34:48 GMT
There's some sort of mismatch:

RegionServer ephemeral node deleted, processing expiration
[localhost.,60020,1295592845214]

and

Waiting on regionserver(s) to go down localhost,60020,1295592845214


Do you see the dot after "localhost" in the first line? I wonder how
it got different in the znode and in ServerManager.onlineServers... In
any case, I'm pretty sure you can get it working by playing with your
/etc/hosts

J-D

On Thu, Jan 20, 2011 at 11:28 PM, James Kennedy
<james.kennedy@troove.net> wrote:
> I've come across a strange bug that I'm having trouble debugging.
> Basically I have a seed application that is executed via maven and runs a
> single JVM ApplicationStarter that starts up hdfs, regionserver, hmaster
> threads. It does some seeding then shuts those down in reverse order.
> So this isn't a typical way of running hbase to be sure. However it has
> always worked until I upgraded to HBase 0.90.0.
> I didn't notice it when I was originally testing 0.90.0 because it only
> seems to be happening on our EC2.small build server node when I run this
> particular seeder.
> Running the same thing locally on my mac works.
> Attached is the error output starting from when the HRegionServer.stop() is
> called to when HMaster.shutdown() is called and it starts looping forever in
> letRegionServersShutdown().
> It looks like RegionServerTracker is getting to "RegionServer ephemeral node
> deleted, processing expiration" but then because it can't get the
> HServerInfo it doesn't follow-through with actually expiring it.
> Does anyone have any ideas as to why this might be happening?
>
>
> Thanks,
> James Kennedy
> Project Manager
> Troove Inc.
>
>

Mime
View raw message