Return-Path: X-Original-To: apmail-accumulo-notifications-archive@minotaur.apache.org Delivered-To: apmail-accumulo-notifications-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 833821779C for ; Fri, 7 Nov 2014 03:22:34 +0000 (UTC) Received: (qmail 70726 invoked by uid 500); 7 Nov 2014 03:22:34 -0000 Delivered-To: apmail-accumulo-notifications-archive@accumulo.apache.org Received: (qmail 70673 invoked by uid 500); 7 Nov 2014 03:22:33 -0000 Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: jira@apache.org Delivered-To: mailing list notifications@accumulo.apache.org Received: (qmail 70662 invoked by uid 99); 7 Nov 2014 03:22:33 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 07 Nov 2014 03:22:33 +0000 Date: Fri, 7 Nov 2014 03:22:33 +0000 (UTC) From: "Josh Elser (JIRA)" To: notifications@accumulo.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (ACCUMULO-3312) IllegalArgumentException in master and gc during randomwalk MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/ACCUMULO-3312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14201518#comment-14201518 ] Josh Elser commented on ACCUMULO-3312: -------------------------------------- Piecing together randomwalk output: table:i is nspc_001.ctt_001 (at least that was its name around the time of the failure) {noformat} 06 21:40:01,837 [concurrent.CloneTable] DEBUG: Cloning table ctt_000 nspc_001.ctt_001 false 06 21:40:55,749 [concurrent.OfflineTable] DEBUG: Offlined nspc_001.ctt_001 06 21:40:56,007 [concurrent.OfflineTable] DEBUG: Onlined nspc_001.ctt_001 {noformat} Looking at the master log gives some more {noformat} 2014-11-06 21:40:58,817 [Audit ] INFO : operation: permitted; user: root; client: 127.0.0.1:35751; action: deleteData; targetTable: nspc_001.ctt_001; startRange: 482773ee160d60e3; endRange: 69dfc517c59b9461; 2014-11-06 21:40:58,854 [zookeeper.DistributedReadWriteLock] INFO : Added lock entry 0 userData 3254d679ddf2a5c7 lockType READ 2014-11-06 21:40:58,854 [tableOps.Utils] INFO : namespace 3 (3254d679ddf2a5c7) locked for read operation: MERGE 2014-11-06 21:40:58,865 [zookeeper.DistributedReadWriteLock] INFO : Added lock entry 0 userData 3254d679ddf2a5c7 lockType WRITE 2014-11-06 21:40:58,866 [tableOps.Utils] INFO : table i (3254d679ddf2a5c7) locked for write operation: MERGE 2014-11-06 21:40:58,870 [master.EventCoordinator] INFO : Merge state of i;69dfc517c59b9461;482773ee160d60e3 set to STARTED 2014-11-06 21:40:58,982 [master.Master] DEBUG: mergeInfo overlaps: i;0886c550cd4b636d< false 2014-11-06 21:40:58,982 [master.Master] DEBUG: mergeInfo overlaps: i;1d3de14e55686f2d;0886c550cd4b636d false 2014-11-06 21:40:58,982 [master.Master] DEBUG: mergeInfo overlaps: i;468075f45a92eebd;1d3de14e55686f2d false 2014-11-06 21:40:58,982 [master.Master] DEBUG: mergeInfo overlaps: i;50a6529a608b142c;468075f45a92eebd true 2014-11-06 21:40:58,982 [master.Master] DEBUG: mergeInfo overlaps: i;60f7b9b99e6b5af9;50a6529a608b142c true 2014-11-06 21:40:58,982 [master.Master] DEBUG: mergeInfo overlaps: i;70c61a87de933d79;60f7b9b99e6b5af9 true 2014-11-06 21:40:58,982 [master.Master] DEBUG: mergeInfo overlaps: i;7540868e587b741b;70c61a87de933d79 false 2014-11-06 21:40:58,982 [master.Master] DEBUG: mergeInfo overlaps: i;77190eca2d9b8b62;7540868e587b741b false 2014-11-06 21:40:58,982 [master.Master] DEBUG: mergeInfo overlaps: i<;77190eca2d9b8b62 false 2014-11-06 21:40:58,983 [state.MergeStats] INFO : Computing next merge state for i;69dfc517c59b9461;482773ee160d60e3 which is presently STARTED isDelete : true 2014-11-06 21:40:58,983 [state.MergeStats] INFO : 3 are hosted, total 3 2014-11-06 21:40:58,983 [state.MergeStats] INFO : Waiting for Merge DELETE of i;69dfc517c59b9461;482773ee160d60e3 State: STARTED lower split to occur i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:40:58,990 [master.EventCoordinator] INFO : Merge state of i;69dfc517c59b9461;482773ee160d60e3 set to SPLITTING 2014-11-06 21:40:59,099 [master.Master] DEBUG: mergeInfo overlaps: i;0886c550cd4b636d< false 2014-11-06 21:40:59,099 [master.Master] DEBUG: mergeInfo overlaps: i;1d3de14e55686f2d;0886c550cd4b636d false 2014-11-06 21:40:59,099 [master.Master] DEBUG: mergeInfo overlaps: i;468075f45a92eebd;1d3de14e55686f2d false 2014-11-06 21:40:59,099 [master.Master] DEBUG: mergeInfo overlaps: i;50a6529a608b142c;468075f45a92eebd true 2014-11-06 21:40:59,099 [master.Master] INFO : Asking localhost:45437[1498722df29016a] to split i;50a6529a608b142c;468075f45a92eebd at 482773ee160d60e3 2014-11-06 21:40:59,272 [master.Master] DEBUG: mergeInfo overlaps: i;60f7b9b99e6b5af9;50a6529a608b142c true 2014-11-06 21:40:59,272 [master.Master] DEBUG: mergeInfo overlaps: i;70c61a87de933d79;60f7b9b99e6b5af9 true 2014-11-06 21:40:59,273 [master.Master] INFO : Asking localhost:45437[1498722df29016a] to split i;70c61a87de933d79;60f7b9b99e6b5af9 at 69dfc517c59b9461 2014-11-06 21:40:59,480 [master.Master] DEBUG: mergeInfo overlaps: i;7540868e587b741b;70c61a87de933d79 false 2014-11-06 21:40:59,480 [master.Master] DEBUG: mergeInfo overlaps: i;77190eca2d9b8b62;7540868e587b741b false 2014-11-06 21:40:59,480 [master.Master] DEBUG: mergeInfo overlaps: i<;77190eca2d9b8b62 false 2014-11-06 21:40:59,481 [state.MergeStats] INFO : Computing next merge state for i;69dfc517c59b9461;482773ee160d60e3 which is presently SPLITTING isDelete : true 2014-11-06 21:40:59,481 [state.MergeStats] INFO : 3 are hosted, total 3 2014-11-06 21:40:59,481 [state.MergeStats] INFO : Waiting for Merge DELETE of i;69dfc517c59b9461;482773ee160d60e3 State: SPLITTING lower split to occur i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:40:59,481 [master.Master] DEBUG: [Normal Tablets] sleeping for 60.00 seconds 2014-11-06 21:40:59,880 [master.EventCoordinator] INFO : localhost:45437 reported split i;482773ee160d60e3;468075f45a92eebd, i;50a6529a608b142c;482773ee160d60e3 2014-11-06 21:40:59,880 [master.EventCoordinator] INFO : localhost:45437 reported split i;69dfc517c59b9461;60f7b9b99e6b5af9, i;70c61a87de933d79;69dfc517c59b9461 2014-11-06 21:40:59,992 [master.Master] DEBUG: mergeInfo overlaps: i;0886c550cd4b636d< false 2014-11-06 21:40:59,993 [master.Master] DEBUG: mergeInfo overlaps: i;1d3de14e55686f2d;0886c550cd4b636d false 2014-11-06 21:40:59,993 [master.Master] DEBUG: mergeInfo overlaps: i;468075f45a92eebd;1d3de14e55686f2d false 2014-11-06 21:40:59,993 [master.Master] DEBUG: mergeInfo overlaps: i;482773ee160d60e3;468075f45a92eebd false 2014-11-06 21:40:59,993 [state.MergeStats] INFO : Lower split found 2014-11-06 21:40:59,993 [master.Master] DEBUG: mergeInfo overlaps: i;50a6529a608b142c;482773ee160d60e3 true 2014-11-06 21:40:59,993 [master.Master] DEBUG: mergeInfo overlaps: i;60f7b9b99e6b5af9;50a6529a608b142c true 2014-11-06 21:40:59,993 [master.Master] DEBUG: mergeInfo overlaps: i;69dfc517c59b9461;60f7b9b99e6b5af9 true 2014-11-06 21:40:59,993 [master.Master] DEBUG: mergeInfo overlaps: i;70c61a87de933d79;69dfc517c59b9461 true 2014-11-06 21:40:59,993 [state.MergeStats] INFO : Upper split found 2014-11-06 21:40:59,993 [master.Master] DEBUG: mergeInfo overlaps: i;7540868e587b741b;70c61a87de933d79 false 2014-11-06 21:40:59,993 [master.Master] DEBUG: mergeInfo overlaps: i;77190eca2d9b8b62;7540868e587b741b false 2014-11-06 21:40:59,993 [master.Master] DEBUG: mergeInfo overlaps: i<;77190eca2d9b8b62 false 2014-11-06 21:40:59,993 [master.EventCoordinator] INFO : [Normal Tablets]: 49 tablets are HOSTED 2014-11-06 21:40:59,993 [master.Master] DEBUG: [Normal Tablets]: scan time 0.01 seconds 2014-11-06 21:40:59,993 [state.MergeStats] INFO : Computing next merge state for i;69dfc517c59b9461;482773ee160d60e3 which is presently SPLITTING isDelete : true 2014-11-06 21:40:59,993 [state.MergeStats] INFO : 4 are hosted, total 4 2014-11-06 21:40:59,993 [state.MergeStats] INFO : 0 tablets are chopped i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:40:59,993 [state.MergeStats] INFO : Waiting for 0 chopped tablets to be 1 i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:41:00,001 [master.EventCoordinator] INFO : Merge state of i;69dfc517c59b9461;482773ee160d60e3 set to WAITING_FOR_CHOPPED 2014-11-06 21:41:00,110 [master.Master] DEBUG: mergeInfo overlaps: i;0886c550cd4b636d< false 2014-11-06 21:41:00,110 [master.Master] DEBUG: mergeInfo overlaps: i;1d3de14e55686f2d;0886c550cd4b636d false 2014-11-06 21:41:00,110 [master.Master] DEBUG: mergeInfo overlaps: i;468075f45a92eebd;1d3de14e55686f2d false 2014-11-06 21:41:00,110 [master.Master] DEBUG: mergeInfo overlaps: i;482773ee160d60e3;468075f45a92eebd false 2014-11-06 21:41:00,110 [state.MergeStats] INFO : Lower split found 2014-11-06 21:41:00,110 [master.Master] DEBUG: mergeInfo overlaps: i;50a6529a608b142c;482773ee160d60e3 true 2014-11-06 21:41:00,110 [master.Master] DEBUG: mergeInfo overlaps: i;60f7b9b99e6b5af9;50a6529a608b142c true 2014-11-06 21:41:00,110 [master.Master] DEBUG: mergeInfo overlaps: i;69dfc517c59b9461;60f7b9b99e6b5af9 true 2014-11-06 21:41:00,110 [master.Master] DEBUG: mergeInfo overlaps: i;70c61a87de933d79;69dfc517c59b9461 true 2014-11-06 21:41:00,110 [state.MergeStats] INFO : Upper split found 2014-11-06 21:41:00,110 [master.Master] INFO : Asking localhost:45437[1498722df29016a] to chop i;70c61a87de933d79;69dfc517c59b9461 2014-11-06 21:41:00,110 [master.Master] DEBUG: mergeInfo overlaps: i;7540868e587b741b;70c61a87de933d79 false 2014-11-06 21:41:00,110 [master.Master] DEBUG: mergeInfo overlaps: i;77190eca2d9b8b62;7540868e587b741b false 2014-11-06 21:41:00,111 [master.Master] DEBUG: mergeInfo overlaps: i<;77190eca2d9b8b62 false 2014-11-06 21:41:00,111 [master.Master] DEBUG: [Normal Tablets]: scan time 0.01 seconds 2014-11-06 21:41:00,111 [state.MergeStats] INFO : Computing next merge state for i;69dfc517c59b9461;482773ee160d60e3 which is presently WAITING_FOR_CHOPPED isDelete : true 2014-11-06 21:41:00,111 [state.MergeStats] INFO : 0 tablets are chopped i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:41:00,111 [state.MergeStats] INFO : Waiting for 0 chopped tablets to be 1 i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:41:00,881 [master.EventCoordinator] INFO : tablet i;70c61a87de933d79;69dfc517c59b9461 chopped 2014-11-06 21:41:00,995 [master.Master] DEBUG: mergeInfo overlaps: i;0886c550cd4b636d< false 2014-11-06 21:41:00,995 [master.Master] DEBUG: mergeInfo overlaps: i;1d3de14e55686f2d;0886c550cd4b636d false 2014-11-06 21:41:00,995 [master.Master] DEBUG: mergeInfo overlaps: i;468075f45a92eebd;1d3de14e55686f2d false 2014-11-06 21:41:00,995 [master.Master] DEBUG: mergeInfo overlaps: i;482773ee160d60e3;468075f45a92eebd false 2014-11-06 21:41:00,995 [state.MergeStats] INFO : Lower split found 2014-11-06 21:41:00,995 [master.Master] DEBUG: mergeInfo overlaps: i;50a6529a608b142c;482773ee160d60e3 true 2014-11-06 21:41:00,995 [master.Master] DEBUG: mergeInfo overlaps: i;60f7b9b99e6b5af9;50a6529a608b142c true 2014-11-06 21:41:00,995 [master.Master] DEBUG: mergeInfo overlaps: i;69dfc517c59b9461;60f7b9b99e6b5af9 true 2014-11-06 21:41:00,995 [master.Master] DEBUG: mergeInfo overlaps: i;70c61a87de933d79;69dfc517c59b9461 true 2014-11-06 21:41:00,995 [state.MergeStats] INFO : Upper split found 2014-11-06 21:41:00,995 [master.Master] DEBUG: mergeInfo overlaps: i;7540868e587b741b;70c61a87de933d79 false 2014-11-06 21:41:00,995 [master.Master] DEBUG: mergeInfo overlaps: i;77190eca2d9b8b62;7540868e587b741b false 2014-11-06 21:41:00,995 [master.Master] DEBUG: mergeInfo overlaps: i<;77190eca2d9b8b62 false 2014-11-06 21:41:00,996 [master.Master] DEBUG: [Normal Tablets]: scan time 0.01 seconds 2014-11-06 21:41:00,996 [master.EventCoordinator] INFO : [Normal Tablets]: 1 tablets unloaded 2014-11-06 21:41:00,996 [state.MergeStats] INFO : Computing next merge state for i;69dfc517c59b9461;482773ee160d60e3 which is presently WAITING_FOR_CHOPPED isDelete : true 2014-11-06 21:41:00,996 [state.MergeStats] INFO : 1 tablets are chopped i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:41:00,996 [state.MergeStats] INFO : 1 tablets are chopped, 0 are offline i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:41:00,996 [state.MergeStats] INFO : Waiting for 0 unassigned tablets to be 4 i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:41:01,001 [master.EventCoordinator] INFO : Merge state of i;69dfc517c59b9461;482773ee160d60e3 set to WAITING_FOR_OFFLINE 2014-11-06 21:41:01,220 [master.Master] DEBUG: mergeInfo overlaps: i;0886c550cd4b636d< false 2014-11-06 21:41:01,220 [master.Master] DEBUG: mergeInfo overlaps: i;1d3de14e55686f2d;0886c550cd4b636d false 2014-11-06 21:41:01,220 [master.Master] DEBUG: mergeInfo overlaps: i;468075f45a92eebd;1d3de14e55686f2d false 2014-11-06 21:41:01,220 [master.Master] DEBUG: mergeInfo overlaps: i;482773ee160d60e3;468075f45a92eebd false 2014-11-06 21:41:01,220 [state.MergeStats] INFO : Lower split found 2014-11-06 21:41:01,220 [master.Master] DEBUG: mergeInfo overlaps: i;50a6529a608b142c;482773ee160d60e3 true 2014-11-06 21:41:01,220 [master.Master] DEBUG: mergeInfo overlaps: i;60f7b9b99e6b5af9;50a6529a608b142c true 2014-11-06 21:41:01,221 [master.Master] DEBUG: mergeInfo overlaps: i;69dfc517c59b9461;60f7b9b99e6b5af9 true 2014-11-06 21:41:01,221 [master.Master] DEBUG: mergeInfo overlaps: i;70c61a87de933d79;69dfc517c59b9461 true 2014-11-06 21:41:01,221 [state.MergeStats] INFO : Upper split found 2014-11-06 21:41:01,221 [master.Master] DEBUG: mergeInfo overlaps: i;7540868e587b741b;70c61a87de933d79 false 2014-11-06 21:41:01,221 [master.Master] DEBUG: mergeInfo overlaps: i;77190eca2d9b8b62;7540868e587b741b false 2014-11-06 21:41:01,221 [master.Master] DEBUG: mergeInfo overlaps: i<;77190eca2d9b8b62 false 2014-11-06 21:41:01,221 [master.EventCoordinator] INFO : [Normal Tablets]: 2 tablets are UNASSIGNED 2014-11-06 21:41:01,221 [master.EventCoordinator] INFO : [Normal Tablets]: 47 tablets are HOSTED 2014-11-06 21:41:01,221 [master.Master] DEBUG: [Normal Tablets]: scan time 0.01 seconds 2014-11-06 21:41:01,221 [master.EventCoordinator] INFO : [Normal Tablets]: 2 tablets unloaded 2014-11-06 21:41:01,221 [state.MergeStats] INFO : Computing next merge state for i;69dfc517c59b9461;482773ee160d60e3 which is presently WAITING_FOR_OFFLINE isDelete : true 2014-11-06 21:41:01,221 [state.MergeStats] INFO : 1 tablets are chopped, 2 are offline i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:41:01,221 [state.MergeStats] INFO : Waiting for 2 unassigned tablets to be 4 i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:41:01,332 [master.Master] DEBUG: mergeInfo overlaps: i;0886c550cd4b636d< false 2014-11-06 21:41:01,332 [master.Master] DEBUG: mergeInfo overlaps: i;1d3de14e55686f2d;0886c550cd4b636d false 2014-11-06 21:41:01,332 [master.Master] DEBUG: mergeInfo overlaps: i;468075f45a92eebd;1d3de14e55686f2d false 2014-11-06 21:41:01,332 [master.Master] DEBUG: mergeInfo overlaps: i;482773ee160d60e3;468075f45a92eebd false 2014-11-06 21:41:01,332 [state.MergeStats] INFO : Lower split found 2014-11-06 21:41:01,332 [master.Master] DEBUG: mergeInfo overlaps: i;50a6529a608b142c;482773ee160d60e3 true 2014-11-06 21:41:01,332 [master.Master] DEBUG: mergeInfo overlaps: i;60f7b9b99e6b5af9;50a6529a608b142c true 2014-11-06 21:41:01,332 [master.Master] DEBUG: mergeInfo overlaps: i;69dfc517c59b9461;60f7b9b99e6b5af9 true 2014-11-06 21:41:01,332 [master.Master] DEBUG: mergeInfo overlaps: i;70c61a87de933d79;69dfc517c59b9461 true 2014-11-06 21:41:01,332 [state.MergeStats] INFO : Upper split found 2014-11-06 21:41:01,332 [master.Master] DEBUG: mergeInfo overlaps: i;7540868e587b741b;70c61a87de933d79 false 2014-11-06 21:41:01,332 [master.Master] DEBUG: mergeInfo overlaps: i;77190eca2d9b8b62;7540868e587b741b false 2014-11-06 21:41:01,332 [master.Master] DEBUG: mergeInfo overlaps: i<;77190eca2d9b8b62 false 2014-11-06 21:41:01,332 [master.EventCoordinator] INFO : [Normal Tablets]: 4 tablets are UNASSIGNED 2014-11-06 21:41:01,332 [master.EventCoordinator] INFO : [Normal Tablets]: 45 tablets are HOSTED 2014-11-06 21:41:01,332 [master.Master] DEBUG: [Normal Tablets]: scan time 0.01 seconds 2014-11-06 21:41:01,332 [state.MergeStats] INFO : Computing next merge state for i;69dfc517c59b9461;482773ee160d60e3 which is presently WAITING_FOR_OFFLINE isDelete : true 2014-11-06 21:41:01,332 [state.MergeStats] INFO : 1 tablets are chopped, 4 are offline i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:41:01,334 [state.MergeStats] DEBUG: Scanning range [i;482773ee160d60e3%00; : [] 9223372036854775807 false,+inf) 2014-11-06 21:41:01,343 [state.MergeStats] DEBUG: consistency check: i;50a6529a608b142c;482773ee160d60e3@(null,null,localhost:45437[1498722df29016a]) walogs 0 2014-11-06 21:41:01,343 [state.MergeStats] DEBUG: consistency check: i;60f7b9b99e6b5af9;50a6529a608b142c@(null,null,localhost:45437[1498722df29016a]) walogs 0 2014-11-06 21:41:01,343 [state.MergeStats] DEBUG: consistency check: i;69dfc517c59b9461;60f7b9b99e6b5af9@(null,null,null) walogs 0 2014-11-06 21:41:01,344 [state.MergeStats] DEBUG: consistency check: i;70c61a87de933d79;69dfc517c59b9461@(null,null,localhost:45437[1498722df29016a]) chopped walogs 0 2014-11-06 21:41:01,344 [state.MergeStats] INFO : Upper split found 2014-11-06 21:41:01,344 [state.MergeStats] DEBUG: consistency check: i;7540868e587b741b;70c61a87de933d79@(null,localhost:45437[1498722df29016a],localhost:45437[1498722df29016a]) walogs 0 2014-11-06 21:41:01,344 [state.MergeStats] DEBUG: chopped 1 v.chopped 1 unassigned 4 v.unassigned 4 verify.total 4 2014-11-06 21:41:01,344 [state.MergeStats] INFO : 4 tablets are unassigned i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:41:01,348 [master.EventCoordinator] INFO : Merge state of i;69dfc517c59b9461;482773ee160d60e3 set to MERGING 2014-11-06 21:41:01,348 [master.Master] DEBUG: Deleting tablets for i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:41:01,350 [master.Master] DEBUG: Found following tablet i;70c61a87de933d79;69dfc517c59b9461 2014-11-06 21:41:01,350 [balancer.DefaultLoadBalancer] DEBUG: balance ended with 0 migrations 2014-11-06 21:41:01,350 [master.Master] DEBUG: Making file deletion entries for i;69dfc517c59b9461;482773ee160d60e3 2014-11-06 21:41:01,350 [balancer.DefaultLoadBalancer] DEBUG: balance ended with 0 migrations 2014-11-06 21:41:01,353 [master.Master] ERROR: Unable merge metadata table records org.apache.accumulo.core.client.AccumuloException: java.lang.IllegalArgumentException: Invalid table path hdfs://promiscuity.penguinsinabox.com:8020/apps/accumulo/tables/ic-00000003 at org.apache.accumulo.master.TabletGroupWatcher.deleteTablets(TabletGroupWatcher.java:558) at org.apache.accumulo.master.TabletGroupWatcher.updateMergeState(TabletGroupWatcher.java:461) at org.apache.accumulo.master.TabletGroupWatcher.run(TabletGroupWatcher.java:294) Caused by: java.lang.IllegalArgumentException: Invalid table path hdfs://promiscuity.penguinsinabox.com:8020/apps/accumulo/tables/ic-00000003 at org.apache.accumulo.server.fs.FileRef.extractSuffix(FileRef.java:71) at org.apache.accumulo.server.fs.FileRef.(FileRef.java:43) at org.apache.accumulo.server.fs.FileRef.(FileRef.java:47) at org.apache.accumulo.master.TabletGroupWatcher.deleteTablets(TabletGroupWatcher.java:520) ... 2 more {noformat} This gets us to the first occurrence of this error. > IllegalArgumentException in master and gc during randomwalk > ----------------------------------------------------------- > > Key: ACCUMULO-3312 > URL: https://issues.apache.org/jira/browse/ACCUMULO-3312 > Project: Accumulo > Issue Type: Bug > Components: gc, master > Affects Versions: 1.6.1 > Reporter: Josh Elser > Assignee: Josh Elser > Priority: Blocker > Fix For: 1.6.2, 1.7.0 > > Attachments: metadata.txt > > > Saw the following: > {noformat} > Unable merge metadata table records > org.apache.accumulo.core.client.AccumuloException: java.lang.IllegalArgumentException: Invalid table path hdfs://promiscuity.penguinsinabox.com:8020/apps/accumulo/tables/ic-00000003 > at org.apache.accumulo.master.TabletGroupWatcher.deleteTablets(TabletGroupWatcher.java:558) > at org.apache.accumulo.master.TabletGroupWatcher.updateMergeState(TabletGroupWatcher.java:461) > at org.apache.accumulo.master.TabletGroupWatcher.run(TabletGroupWatcher.java:294) > Caused by: java.lang.IllegalArgumentException: Invalid table path hdfs://promiscuity.penguinsinabox.com:8020/apps/accumulo/tables/ic-00000003 > at org.apache.accumulo.server.fs.FileRef.extractSuffix(FileRef.java:71) > at org.apache.accumulo.server.fs.FileRef.(FileRef.java:43) > at org.apache.accumulo.server.fs.FileRef.(FileRef.java:47) > at org.apache.accumulo.master.TabletGroupWatcher.deleteTablets(TabletGroupWatcher.java:520) > ... 2 more > {noformat} > {noformat} > java.lang.IllegalArgumentException: hdfs://promiscuity.penguinsinabox.com:8020/apps/accumulo/tables/ic-00000000 > java.lang.IllegalArgumentException: hdfs://promiscuity.penguinsinabox.com:8020/apps/accumulo/tables/ic-00000000 > at org.apache.accumulo.gc.GarbageCollectionAlgorithm.makeRelative(GarbageCollectionAlgorithm.java:94) > at org.apache.accumulo.gc.GarbageCollectionAlgorithm.confirmDeletes(GarbageCollectionAlgorithm.java:199) > at org.apache.accumulo.gc.GarbageCollectionAlgorithm.confirmDeletesTrace(GarbageCollectionAlgorithm.java:249) > at org.apache.accumulo.gc.GarbageCollectionAlgorithm.collect(GarbageCollectionAlgorithm.java:284) > at org.apache.accumulo.gc.SimpleGarbageCollector.run(SimpleGarbageCollector.java:549) > at org.apache.accumulo.gc.SimpleGarbageCollector.main(SimpleGarbageCollector.java:154) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.accumulo.start.Main$1.run(Main.java:141) > at java.lang.Thread.run(Thread.java:745) > {noformat} > Running randomwalk on a single node with a 2nd tserver started. Ran ShortClean.xml, specifically concurrent, then image (errors occurred during Image, not sure if it was from somethign that happened during concurrent though). -- This message was sent by Atlassian JIRA (v6.3.4#6332)