hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Samir Ahmic <ahmic.sa...@gmail.com>
Subject Re: Phantom region server and PENDING_OPEN regions
Date Tue, 24 Nov 2015 09:23:27 GMT
Hi Kristoffer,
It looks like you have some issue with name resolution. Try to remove
incorrect value from reslove.conf (node.dc1.consul) and then restart hbase
cluster.
Regarding issue with region in transition check master log for
"hbase:meta,,1.1588230740"
there should be exception explaining why hbase:meta can to be transition
from PENDING_OPEN to OPEN state, if hbase:meta table is unavailable master
can not finish initialization.

Regards
Samir

On Tue, Nov 24, 2015 at 10:11 AM, Kristoffer Sjögren <stoffe@gmail.com>
wrote:

> Sorry, I should mention that this is HBase 1.1.2.
>
> Zookeeper only report one region server.
>
> $ ls /hbase-unsecure/rs
> [amb2.service.consul,16020,1448353564099]
>
>
>
>
> On Tue, Nov 24, 2015 at 9:55 AM, Kristoffer Sjögren <stoffe@gmail.com>
> wrote:
> > Hi
> >
> > I'm trying to install a HBase cluster with 1 master
> > (amb1.service.consul) and 1 region server (amb2.service.consul) using
> > Ambari on docker containers provided by sequenceiq [1] using a custom
> > blueprint [2].
> >
> > Every component installs correctly except for HBase which get stuck
> > with regions in transition:
> >
> > ---
> > hbase:meta,,1.1588230740 state=PENDING_OPEN, ts=Tue Nov 24 08:26:45
> > UTC 2015 (1098s ago), server=amb2.service.consul,16020,1448353564099
> > ---
> >
> > And for some reason 2 region servers (instead of 1) are discovered by
> > the master with the exact same timestamp but with different hostnames.
> > I'm not sure if this is the reason why the regions get stuck.
> >
> > ----
> > amb2.node.dc1.consul,16020,1448353564099Tue Nov 24 08:26:04 UTC 201500
> > amb2.service.consul,16020,1448353564099Tue Nov 24 08:26:04 UTC 201500
> > ----
> >
> > The only place I can find "amb2.node.dc1.consul" on the ambari
> > agent/server hosts is in /etc/resolv.conf which looks like this.
> >
> > ----
> > nameserver 172.17.0.82
> > search service.consul node.dc1.consul
> > ----
> >
> > Is there some way that I can manually tell the master to disregard the
> > "phantom" host amb2.node.dc1.consul?
> >
> > Any help or tips appreciated.
> >
> > Cheers,
> > -Kristoffer
> >
> >
> > [1] https://github.com/sequenceiq/docker-ambari
> > [2]
> https://gist.githubusercontent.com/krisskross/901ed8223c1ed1db80e3/raw/869327be9ad15e6a9f099a7591323244cd245357/ambari-hdp2.3
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message