hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: a table is neither disable or enable
Date Wed, 29 Aug 2018 23:00:00 GMT
I doubt the procedure would finish, considering it has run for so long.

You can check the tail of master log to see if it is stuck.
If it is stuck, see if you can use abort_procedure.rb to stop.

After the procedure is stopped, see if running hbck can fix the issue (I
haven't worked with 1.3 release in production).
When running hbck, run without -fix parameter first to see what
inconsistencies hbck reports.

Cheers

On Wed, Aug 29, 2018 at 3:42 PM Antonio Si <antonio.si@gmail.com> wrote:

> Forgot to mention that all regions of the table is offline now. Wondering
> if the table will eventually got disable as it has been running for almost
> 24 hrs now.
>
> Thanks.
>
> Antonio.
>
> On Wed, Aug 29, 2018 at 3:40 PM Antonio Si <antonio.si@gmail.com> wrote:
>
> > Thanks Ted.
> > Now that the table is in neither disable or enable state, will the table
> > eventually got disable completely?
> > From the "Procedure" tab of the hbase ui, I see the "disable" is still
> > running.
> >
> > Thanks.
> >
> > Antonio.
> >
> > On Wed, Aug 29, 2018 at 3:31 PM Ted Yu <yuzhihong@gmail.com> wrote:
> >
> >> The 'missing table descriptor' error should have been fixed by running
> >> hbck
> >> (with selected parameters).
> >>
> >> FYI
> >>
> >> On Wed, Aug 29, 2018 at 2:46 PM Antonio Si <antonio.si@gmail.com>
> wrote:
> >>
> >> > Thanks Ted.
> >> >
> >> > The log says "java.io.IOException: missing table descriptor for
> >> > ba912582f295f7ac0b83e7e419351602
> >> >
> >> > [AM.ZK.Worker-pool2-t6552] master.RegionStates: Failed to open/close
> >> > ba912582f295f7ac0b83e7e419351602  set to FAILED_OPEN"
> >> >
> >> >
> >> > The version of hbase is 1.3.1
> >> >
> >> >
> >> > Thanks.
> >> >
> >> >
> >> > Antonio.
> >> >
> >> > On Wed, Aug 29, 2018 at 2:28 PM Ted Yu <yuzhihong@gmail.com> wrote:
> >> >
> >> > > Do you have access to master / region logs for when FAILED_OPEN
> state
> >> was
> >> > > noticed ?
> >> > >
> >> > > There should be some hint there as to why some region couldn't open.
> >> > >
> >> > > The length of table DDL is related to number of regions the table
> has.
> >> > But
> >> > > the length should be less related to data amount.
> >> > >
> >> > > Which version of hbase are you using ?
> >> > >
> >> > > Thanks
> >> > >
> >> > > On Wed, Aug 29, 2018 at 2:22 PM Antonio Si <antonio.si@gmail.com>
> >> wrote:
> >> > >
> >> > > > Hi,
> >> > > >
> >> > > > We have a table which is stuck in FAILED_OPEN state. So, we
> planned
> >> to
> >> > > drop
> >> > > > the table and re-clone the table from an old snapshot. We disabled
> >> the
> >> > > > table, but the disable procedure has been running for more than
20
> >> hrs.
> >> > > >
> >> > > > I went to hbase shell and found out "is_disabled" and "is_enabled"
> >> both
> >> > > > return false. Is that a normal behavior since the table is in
the
> >> > middle
> >> > > of
> >> > > > being disabled?
> >> > > >
> >> > > > Is it normal that the disable took that many hours even though
the
> >> > table
> >> > > is
> >> > > > large in size (about 33TB)?
> >> > > >
> >> > > > Thanks.
> >> > > >
> >> > > > Antonio.
> >> > > >
> >> > >
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message