Return-Path: X-Original-To: apmail-accumulo-notifications-archive@minotaur.apache.org Delivered-To: apmail-accumulo-notifications-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id AC41B10F4B for ; Thu, 20 Mar 2014 16:33:44 +0000 (UTC) Received: (qmail 35967 invoked by uid 500); 20 Mar 2014 16:33:31 -0000 Delivered-To: apmail-accumulo-notifications-archive@accumulo.apache.org Received: (qmail 35763 invoked by uid 500); 20 Mar 2014 16:33:20 -0000 Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: jira@apache.org Delivered-To: mailing list notifications@accumulo.apache.org Received: (qmail 35195 invoked by uid 99); 20 Mar 2014 16:32:59 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 20 Mar 2014 16:32:59 +0000 Date: Thu, 20 Mar 2014 16:32:59 +0000 (UTC) From: "Sean Busbey (JIRA)" To: notifications@accumulo.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (ACCUMULO-2361) droptable created infinite METADATA scan loop MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/ACCUMULO-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Busbey updated ACCUMULO-2361: ---------------------------------- Fix Version/s: 1.6.0 > droptable created infinite METADATA scan loop > --------------------------------------------- > > Key: ACCUMULO-2361 > URL: https://issues.apache.org/jira/browse/ACCUMULO-2361 > Project: Accumulo > Issue Type: Bug > Affects Versions: 1.5.1 > Environment: accumulo-1.5.0 with sqrrl extensions. > Reporter: Chris McCubbin > Assignee: Eric Newton > Priority: Critical > Fix For: 1.6.0 > > Attachments: AccumuloHammerTableCreation.java, AccumuloHammerTableCreation.java, Screen Shot 2014-02-12 at 2.27.11 PM.png, Screen Shot 2014-02-12 at 2.28.09 PM.png, TableCreationHammerIT.java, jstack1.txt, jstack2.txt, jstack3.txt, masterJstack.txt, masterJstack2.txt, masterJstack3.txt, short.debug.log > > > Working with [~vines] on this one... > Setup: Created a couple tables, added some data, then dropped them. The drop hangs and !METADATA (which has ~400 entries) is scanned in what looks like an infinite loop. > The table being dropped loks like this in !METADATA: > {code} > root@sqrrl> scan -b 3 -e 5 -t !METADATA > 4;\x00\x00\x06f srv:dir [] /t-00000b6 > 4;\x00\x00\x06f srv:lock [] tservers/10.10.1.209:9997/zlock-0000000000$144274cd317000b > 4;\x00\x00\x06f srv:time [] M0 > 4;\x00\x00\x06f ~tab:~pr [] \x00 > 4;\x00\x00\x0C\xCC loc:144274cd3170008 [] 10.10.1.107:9997 > 4;\x00\x00\x0C\xCC srv:dir [] /t-00000bj > 4;\x00\x00\x0C\xCC srv:lock [] tservers/10.10.1.209:9997/zlock-0000000000$144274cd317000b > 4;\x00\x00\x0C\xCC srv:time [] M0 > 4;\x00\x00\x0C\xCC ~tab:~pr [] \x01\x00\x00\x06f > 4;\x00\x00\x133 srv:dir [] /t-000002h > 4;\x00\x00\x133 srv:lock [] tservers/10.10.1.209:9997/zlock-0000000000$144274cd317000b > 4;\x00\x00\x133 srv:time [] M0 > 4;\x00\x00\x133 ~tab:~pr [] \x01\x00\x00\x0C\xCC > {code} > We think this may be the relevant message in the master debug logs: > {code} > 2014-02-12 19:13:31,397 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted: 4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff > 2014-02-12 19:13:31,459 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted: 4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff > 2014-02-12 19:13:31,524 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted: 4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff > 2014-02-12 19:13:31,588 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted: 4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff > 2014-02-12 19:13:31,662 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted: 4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff > 2014-02-12 19:13:31,725 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted: 4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff > 2014-02-12 19:13:31,788 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted: 4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff > 2014-02-12 19:13:31,854 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted: 4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff > 2014-02-12 19:13:31,917 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted: 4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff > ...etc > {code} > Graceful accumulo reboot hangs. > Hard reboot of everything (control-c'd) clears the problem. -- This message was sent by Atlassian JIRA (v6.2#6252)