hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Busbey <bus...@apache.org>
Subject Re: 2.0.4 to 2.2.0 testing
Date Tue, 12 Mar 2019 22:17:49 GMT
On Tue, Mar 12, 2019 at 2:21 PM Jean-Marc Spaggiari
<jean-marc@spaggiari.org> wrote:
>
> > The important bit will be that you get a currently working HBCK2 instance
>
> I did. I downloaded all 2.2.0 artifacts locally, and build with the
> parameters you gave. All went well. However, HBCK2 can't talk to my HBase
> server...
>
> I tried this:
> java org.apache.hbase.HBCK2 assigns 1431292037213
>
> But I got this:
> PleaseHoldException: Master is initializing
>

That's not how HBCK2 is invoked. Let's get the correct invocation
working and then we can work out what's going on with master.


> The hbase hbck -j ~/hbase-hbck2-for-2.2.0.jar unassigns 1588230740 command
> doesn.t seems to be correct. "-j" is not a valid parameter, should be -jar.
> And fixing it displayes the previous HBCK version help.
>

Okay, something is still wrong then. The cli option is definitely "-j"
and not "-jar".

Which "hbase" command is being run? Can you confirm it's the one in
the bin tarball? what does the version command output? What does the
mapredcp command output? What does the cli help for the hbase command
show?



> Because the META is now assigned, I changed 1588230740 to 1431292037213 to
> get the namespace table assigned. No idea how the meta got assigned... I
> removed /hbase from ZK, maybe it helped.
>

It's not good that we're bouncing around states without knowing why
those states are changing. Let's focus on getting hbck2 invocations to
work as expected before continuing with cluster health.

> Cluster is still running in maintenance mode:
>   <property>
>     <name>hbase.master.maintenance_mode</name>
>     <value>true</value>
>   </property>
>

I don't recall suggesting maintenance mode and would suggest not being
in it for now.

> That's why I'm very surprised it tries to still assign some regions to
> other servers (master is node2)
> 2019-03-12 10:59:12,286 WARN  [master/node2:60000:becomeActiveMaster]
> master.HMaster:
> hbase:namespace,,1431292037213.7f4a480f47f98300185d1ae2ff663295. is NOT
> online; state={7f4a480f47f98300185d1ae2ff663295 state=OPENING,
> ts=1552402749342, server=node3,16020,1551984565705};
> ServerCrashProcedures=true. Master startup cannot progress, in
> holding-pattern until region onlined.
>

Yes, that is unusual. Let's maybe come back to that if necessary, but
for now please stop trying to use maintenance mode.

Mime
View raw message