accumulo-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Wall <mjw...@gmail.com>
Subject Re: How can I remove or fix a hanging droptable in Accumulo?
Date Thu, 13 Oct 2016 00:27:24 GMT
Jeff,

There was a race condition around splitting I think, but I can't find the
issue right now.  I caught a stacktrace at one point doing just as Ed
suggests, then setting 'debug on' in the shell try the droptable again.
After I found the issue, I turned off all ingest and waited for the master
stop balancing and tables to stop splitting.  My droptable was successfully
issued after that (doing Ed's trick once again).

I can look for that issue tomorrow or if you catch a stacktrace, post it
here.

Mike

On Wed, Oct 12, 2016 at 7:57 PM, Ed Coleman <dev1@etcoleman.com> wrote:

> This is a brute force approach and should only done as a last resort.
> Sometimes bouncing the master(s) helps – make sure all of stopped, not just
> failed over to another instance., but it sounds like you have tried that.
>
>
>
> You may be able to force it by manually editing zookeeper  nodes using
> zkCli.sh You are editing zookeeper so make sure you have the correct table
> id and path(s)
>
>
>
> Add permission to write in zookeeper using:
>
>
>
> zkCli> addauth digest accumulo:YOUR_SECRET
>
>
>
> Under /accumulo/INSTANCE_ID/tables/TABLE_ID/state (going from memory,
> could be fate) you can use examine the state
>
>
>
> zkCli> ls /accumulo/INSTANCE_ID/tables/TABLE_ID/
>
>
>
> there should be mutlple zk nodes, one of which is state (or maybe fate?)
>
>
>
> zkCli> get /accumulo/INSTANCE_ID/tables/TABLE_ID/state
>
>
>
> – it should say DELETING.
>
>
>
> zkCli> set /accumulo/INSTANCE_ID/tables/TABLE_ID/state  ONLINE
>
>
>
> This will force the table back online. After all tablets are accounted for
> in the monitor you should be able to re-issue the delete command via the
> accumulo shell.
>
>
>
> Ed Coleman
>
>
>
> *From:* Jeff Kubina [mailto:jeff.kubina@gmail.com]
> *Sent:* Wednesday, October 12, 2016 7:38 PM
> *To:* user@accumulo.apache.org
> *Subject:* Re: How can I remove or fix a hanging droptable in Accumulo?
>
>
>
> Lots of bulk ingest assignments. It's been hung for days and even returned
> after a reboot of the master and tservers.
>
>
> --
>
> Jeff Kubina
> 410-988-4436
>
>
>
>
>
> On Wed, Oct 12, 2016 at 6:28 PM, Michael Wall <mjwall@gmail.com> wrote:
>
> Jeff,
>
> What is the master doing while the droptable is hung?
>
> Mike
>
>
>
> On Wed, Oct 12, 2016 at 6:14 PM, Jeff Kubina <jeff.kubina@gmail.com>
> wrote:
>
> How can I remove or fix a hanging droptable in Accumulo?
>
>
>
>
>

Mime
View raw message