accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-2361) droptable crated infinite METADATA scan loop
Date Wed, 12 Feb 2014 22:20:19 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13899670#comment-13899670
] 

Eric Newton commented on ACCUMULO-2361:
---------------------------------------

Strange.  The master is seeing the first tablet as: {{4;\x00\x00\x0c?;\x00\x00\x06f}} and
the first tablet is {{4;\x00\x00\x0c?<}}.  It could be confused by a recent split, but
eventually it should see the METADATA table scan you included above (which looks correct).
 Maybe there's some caching that isn't getting reset because we're not trying to find the
tablets?




> droptable crated infinite METADATA scan loop
> --------------------------------------------
>
>                 Key: ACCUMULO-2361
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2361
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.5.0
>            Reporter: Chris McCubbin
>
> Working with [~vines] on this one...
> Setup: Created a couple tables, added some data, then dropped them. The drop hangs and
!METADATA (which has ~400 entries) is scanned in what looks like an infinite loop.
> The table being dropped loks like this in !METADATA:
> {code}
> root@sqrrl> scan -b 3 -e 5 -t !METADATA
> 4;\x00\x00\x06f srv:dir []    /t-00000b6
> 4;\x00\x00\x06f srv:lock []    tservers/10.10.1.209:9997/zlock-0000000000$144274cd317000b
> 4;\x00\x00\x06f srv:time []    M0
> 4;\x00\x00\x06f ~tab:~pr []    \x00
> 4;\x00\x00\x0C\xCC loc:144274cd3170008 []    10.10.1.107:9997
> 4;\x00\x00\x0C\xCC srv:dir []    /t-00000bj
> 4;\x00\x00\x0C\xCC srv:lock []    tservers/10.10.1.209:9997/zlock-0000000000$144274cd317000b
> 4;\x00\x00\x0C\xCC srv:time []    M0
> 4;\x00\x00\x0C\xCC ~tab:~pr []    \x01\x00\x00\x06f
> 4;\x00\x00\x133 srv:dir []    /t-000002h
> 4;\x00\x00\x133 srv:lock []    tservers/10.10.1.209:9997/zlock-0000000000$144274cd317000b
> 4;\x00\x00\x133 srv:time []    M0
> 4;\x00\x00\x133 ~tab:~pr []    \x01\x00\x00\x0C\xCC
> {code}
> We think this may be the relevant message in the master debug logs:
> {code}
> 2014-02-12 19:13:31,397 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted:
4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff
> 2014-02-12 19:13:31,459 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted:
4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff
> 2014-02-12 19:13:31,524 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted:
4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff
> 2014-02-12 19:13:31,588 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted:
4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff
> 2014-02-12 19:13:31,662 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted:
4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff
> 2014-02-12 19:13:31,725 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted:
4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff
> 2014-02-12 19:13:31,788 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted:
4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff
> 2014-02-12 19:13:31,854 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted:
4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff
> 2014-02-12 19:13:31,917 [tableOps.CleanUp] DEBUG: Still waiting for table to be deleted:
4 saw inconsistencynull 4;^@^@^L?;^@^@^Ff
> ...etc
> {code}
> Graceful accumulo reboot hangs. 
> Hard reboot of everything (control-c'd) clears the problem.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message