zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Huttenhuis (Jira)" <j...@apache.org>
Subject [jira] [Created] (ZOOKEEPER-3513) Zookeeper upgrade fails due to missing snapshots
Date Wed, 21 Aug 2019 08:19:00 GMT
Stephan Huttenhuis created ZOOKEEPER-3513:
---------------------------------------------

             Summary: Zookeeper upgrade fails due to missing snapshots
                 Key: ZOOKEEPER-3513
                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3513
             Project: ZooKeeper
          Issue Type: Bug
          Components: server
    Affects Versions: 3.5.4, 3.6.0
            Reporter: Stephan Huttenhuis


In ZOOKEEPER-2325 a check was added that requires a snapshot when loading data. We have been
running 3-node ensembles on Zookeeper 3.4.13 for about 5 months for use with Solr Cloud. During
this time some ensembles created a few snapshots but other didn't generate any. Because of
this upgrading to e.g. 3.5.5 fails.

Either it is perfectly possible for Zookeeper data to have no snapshots or something is going
wrong with generating snapshots. The ensembles are straightforward.
 - The following stack occurs:
{noformat}
java.io.IOException: No snapshot found, but there are log entries. Something is broken!
	at org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:211)
	at org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:240)
	at org.apache.zookeeper.server.ZooKeeperServer.loadData(ZooKeeperServer.java:290)
	at org.apache.zookeeper.server.ZooKeeperServer.startdata(ZooKeeperServer.java:450)
	at org.apache.zookeeper.server.NIOServerCnxnFactory.startup(NIOServerCnxnFactory.java:764)
	at org.apache.zookeeper.server.ServerCnxnFactory.startup(ServerCnxnFactory.java:98)
	at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:144)
	at org.apache.zookeeper.server.ZooKeeperServerMain.initializeAndRun(ZooKeeperServerMain.java:106)
	at org.apache.zookeeper.server.ZooKeeperServerMain.main(ZooKeeperServerMain.java:64)
	at org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:128)
	at org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:82)
{noformat}

 - The zoo.cfg
{noformat}
# The number of milliseconds of each tick
tickTime=2000
# The number of ticks that the initial
# synchronization phase can take
initLimit=10
# The number of ticks that can pass between
# sending a request and getting an acknowledgement
syncLimit=5
# the directory where the snapshot is stored.
# do not use /tmp for storage, /tmp here is just
# example sakes.
dataDir=/data/zookeeper/data
# the port at which the clients will connect
clientPort=2181

server.1=myserver1:2888:3888
server.2=myserver1:2888:3888
server.3=myserver1:2888:3888
{noformat}
 

 - The contents of /data/zookeeper/data/version-2
{noformat}
-rw-r--r-- 1 zookeeper zookeeper    1 Aug  7 21:50 acceptedEpoch
-rw-r--r-- 1 zookeeper zookeeper    1 Aug  8 20:38 currentEpoch
-rw-r--r-- 1 zookeeper zookeeper  65M Apr  1 14:44 log.1
-rw-r--r-- 1 zookeeper zookeeper  65M May 15 23:30 log.100000001
-rw-r--r-- 1 zookeeper zookeeper  65M Jul  3 23:21 log.100001645
-rw-r--r-- 1 zookeeper zookeeper  65M Aug  8 20:37 log.300000802
-rw-r--r-- 1 zookeeper zookeeper  65M Aug 20 13:58 log.70000062a
-rw-r--r-- 1 zookeeper zookeeper  65M Apr  4 21:22 log.f0
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.2#803003)

Mime
View raw message