hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ramkrishna vasudevan <ramkrishna.s.vasude...@gmail.com>
Subject Re: Re: Re: Wrong Configuration lead to a failure when enabling table
Date Wed, 04 Feb 2015 05:01:57 GMT
It is not only about the state on the table descriptor but also the in
memory state in the AM.  I remember some time back Rajeshbabu worked on a
HBCK like tool which will forcefully change the state of these tables in
such cases. I don't remember the JIRA now.I thought of restarting the
master thinking the in memory state would change and I got this

java.lang.NullPointerException
        at
org.apache.hadoop.hbase.master.handler.EnableTableHandler.handleEnableTable(EnableTableHandler.java:210)
        at
org.apache.hadoop.hbase.master.handler.EnableTableHandler.process(EnableTableHandler.java:142)
        at
org.apache.hadoop.hbase.master.AssignmentManager.recoverTableInEnablingState(AssignmentManager.java:1695)
        at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:416)
        at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:720)
        at
org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:170)
        at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1459)
        at java.lang.Thread.run(Thread.java:745)
2015-02-04 16:11:45,932 FATAL [stobdtserver3:16040.activeMasterManager]
master.HMaster: Master server abort: loaded coprocessors are:
[org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint]
2015-02-04 16:11:45,933 FATAL [stobdtserver3:16040.activeMasterManager]
master.HMaster: Unhandled exception. Starting shutdown.
java.lang.NullPointerException
        at
org.apache.hadoop.hbase.master.handler.EnableTableHandler.handleEnableTable(EnableTableHandler.java:210)
        at
org.apache.hadoop.hbase.master.handler.EnableTableHandler.process(EnableTableHandler.java:142)
        at
org.apache.hadoop.hbase.master.AssignmentManager.recoverTableInEnablingState(AssignmentManager.java:1695)
        at
org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:416)
        at
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:720)
        at
org.apache.hadoop.hbase.master.HMaster.access$500(HMaster.java:170)
        at org.apache.hadoop.hbase.master.HMaster$1.run(HMaster.java:1459)


Regards
Ram

On Wed, Feb 4, 2015 at 10:25 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> What about creating an offline tool which can modify the table descriptor
> so that table goes to designated state ?
>
> Cheers
>
> On Tue, Feb 3, 2015 at 8:51 PM, ramkrishna vasudevan <
> ramkrishna.s.vasudevan@gmail.com> wrote:
>
> > I tried reproducing this scenario on trunk. The same problem exists.
> > Currently in the master the table state is noted in the Table descriptor
> > and not on the ZK.  In 0.98.XX version it should be on the zk.
> >
> > When we tried to enable the table the region assignment failed due to
> > ClassNotFound and already the state is in ENABLING.  But doing a describe
> > table still shows it in DISABLED.
> >
> > Thought we could alter the correct Configuration but specifying another
> > alter Table command we are still not able to enable the table.
> >
> > Moving this to dev to see if there is any workaround for this issue.  If
> > not we may have to solve this issue across branches until we have the
> > Procedure V2 implemenation ready on trunk.
> >
> > Any suggestions?
> >
> > Regards
> > Ram
> >
> > On Wed, Feb 4, 2015 at 4:05 AM, 叶炜晨 <yeweichen@qiyi.com> wrote:
> >
> > >  my version is 0.98.6-cdh5.2.0, the problem in my production
> environment.
> > >
> > > So should I first delete znode? And then how to distable this table?my
> > > goal is to fix the wrong table configuration to get my data.
> > >
> > >
> > >  from my mobile phone.
> > >
> > >  在 2015-2-4 上午12:46,ramkrishna vasudevan <
> > ramkrishna.s.vasudevan@gmail.com
> > > >写道:
> > >
> > > >
> > > > I think the only way out here is to clear the zookeeper node.  But am
> > > not sure on the ramifications of that.
> > > >
> > >
> > > > Which version are you using?  The newer versions are 'protobuf'fed.
> > > >
> > >
> > > > Are you running this in production?
> > > >
> > >
> > > > Regards
> > > > Ram
> > > >
> > >
> > > > On Tue, Feb 3, 2015 at 5:00 PM,yeweichen2010@gmail.com<
> > > yeweichen2010@gmail.com>wrote:
> > >
> > > >>
> > > >> I tried HBCK, but it doesn`t help.
> > >
> > > I want to disable the table, so that I can use "alter" to fix the wrong
> > > configuration. But now the table keep in the status that no matter I
> use
> > > "is_enabled" or "is_disabled", it return false.
> > > >>
> > > >> ________________________________
> > > >> yeweichen2010@gmail.com
> > > >>>
> > > >>>
> > > >>> From: ramkrishna vasudevan
> > > >>> Date: 2015-02-03 19:55
> > > >>> To: user@hbase.apache.org
> > > >>> CC: yeweichen
> > > >>> Subject: Re: Wrong Configuration lead to a failure when enabling
> > table
> > > >>> Can you try HBCK? Did it help in anyway?  Remember something was
> done
> > > >>> related to failure in ENABLE/DISABLE table some time back.
> > > >>>
> > > >>> Regards
> > > >>> Ram
> > > >>>
> > > >>> On Tue, Feb 3, 2015 at 3:38 PM,yeweichen2010@gmail.com<
> > > >>> yeweichen2010@gmail.com> wrote:
> > > >>>
> > > >>> > Hi, all,
> > > >>> >
> > > >>> > II did the following command in hbase shell:
> > > >>> >
> > > >>> > disable 'TestTable'
> > > >>> > alter 'TestTable', CONFIGURATION =>
> > > >>> > {'hbase.regionserver.region.split.policy' => 'xxxxxxxxx'}
> > > >>> > enable 'TestTable'
> > > >>> >
> > > >>> > At first I want to put
> > > >>> >
> > "org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy"
> > > to the
> > > >>> > place "xxxxxxxxx", but because a spelling error, now is something
> > > wrong in
> > > >>> > this configuration. After I enable the table, it failed bacause
> of
> > > >>> > ClassNotFound.
> > > >>> >
> > > >>> > Now is the problem: the table failed to enable and stay in
a
> middle
> > > >>> > status. The table is neither enabled nor disabled now. How
can I
> > > save my
> > > >>> > table and fix the wrong configuration?
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> >
> > > >>> >yeweichen2010@gmail.com
> > > >>> >
> > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message