accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Newton (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (ACCUMULO-1995) TServer refuses to start if WALog file has only header and no entries
Date Wed, 12 Nov 2014 19:03:34 GMT

     [ https://issues.apache.org/jira/browse/ACCUMULO-1995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eric Newton resolved ACCUMULO-1995.
-----------------------------------
       Resolution: Duplicate
    Fix Version/s: 1.6.0

I missed this ticket when I filled ACCUMULO-2168.

> TServer refuses to start if WALog file has only header and no entries
> ---------------------------------------------------------------------
>
>                 Key: ACCUMULO-1995
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1995
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.6.0
>            Reporter: Michael Allen
>             Fix For: 1.6.0
>
>         Attachments: ACCUMULO-1995-Fixes-ZK-path-construction.patch
>
>
> In the course of exercising Accumulo 1.6.0 with encryption turned on, we were able to
get tablet servers into the following state many times, wherein the tablet server would not
start up and instead just print this over and over to the log:
> {noformat}
> 2013-12-09 14:13:04,487 [util.MetadataTableUtil] ERROR: java.lang.IllegalArgumentException:
Invalid path string "/accumulo/0147d545-64bd-4383-b842-27c62283c208/root_tablet/walogs/hdfs://10.10.1.10:9000/accumulo/wal/10.10.1.10+9997/e42efc86-6d83-4c0e-abd6-b16ec74d0a9f"
caused by empty node name specified @72
> java.lang.IllegalArgumentException: Invalid path string "/accumulo/0147d545-64bd-4383-b842-27c62283c208/root_tablet/walogs/hdfs://10.10.1.10:9000/accumulo/wal/10.10.1.10+9997/e42efc86-6d83-4c0e-abd6-b16ec74d0a9f"
caused by empty node name specified @72
> 	at org.apache.zookeeper.common.PathUtils.validatePath(PathUtils.java:99)
> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1450)
> 	at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1496)
> 	at org.apache.accumulo.fate.zookeeper.ZooUtil.recursiveDelete(ZooUtil.java:103)
> 	at org.apache.accumulo.fate.zookeeper.ZooUtil.recursiveDelete(ZooUtil.java:117)
> 	at org.apache.accumulo.fate.zookeeper.ZooReaderWriter.recursiveDelete(ZooReaderWriter.java:64)
> 	at org.apache.accumulo.server.util.MetadataTableUtil.removeUnusedWALEntries(MetadataTableUtil.java:611)
> 	at org.apache.accumulo.tserver.Tablet.<init>(Tablet.java:1394)
> 	at org.apache.accumulo.tserver.Tablet.<init>(Tablet.java:1236)
> 	at org.apache.accumulo.tserver.Tablet.<init>(Tablet.java:1091)
> 	at org.apache.accumulo.tserver.Tablet.<init>(Tablet.java:1079)
> 	at org.apache.accumulo.tserver.TabletServer$AssignmentHandler.run(TabletServer.java:2892)
> 	at org.apache.accumulo.core.util.LoggingRunnable.run(LoggingRunnable.java:34)
> 	at org.apache.accumulo.tserver.TabletServer$ThriftClientHandler$3.run(TabletServer.java:2261)
> {noformat}
> The log file in question had a good header but no entries in it, probably the result
of killing a tablet server before it had any data to write.  The bug in the code has to do
with how log file names are now held within ZooKeeper (they have a full path) versus how the
paths to their metadata within  ZooKeeper are eventually constructed by {{MetadataTableUtil}}.
 The fix is relatively straightforward; you just need to take apart the full HDFS path before
appending it to a different ZooKeeper path.
> Patch coming.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message