zookeeper-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andor Molnar (Jira)" <j...@apache.org>
Subject [jira] [Resolved] (ZOOKEEPER-3644) Data loss after upgrading standalone ZK server 3.4.14 to 3.5.6 with snapshot.trust.empty=true
Date Sun, 05 Jan 2020 21:20:00 GMT

     [ https://issues.apache.org/jira/browse/ZOOKEEPER-3644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andor Molnar resolved ZOOKEEPER-3644.
-------------------------------------
    Fix Version/s: 3.7.0
                   3.5.7
                   3.6.0
       Resolution: Fixed

Issue resolved by pull request 1182
[https://github.com/apache/zookeeper/pull/1182]

> Data loss after upgrading standalone ZK server 3.4.14 to 3.5.6 with snapshot.trust.empty=true
> ---------------------------------------------------------------------------------------------
>
>                 Key: ZOOKEEPER-3644
>                 URL: https://issues.apache.org/jira/browse/ZOOKEEPER-3644
>             Project: ZooKeeper
>          Issue Type: Bug
>          Components: server
>    Affects Versions: 3.5.6
>            Reporter: Manikumar
>            Assignee: Michael Han
>            Priority: Blocker
>              Labels: pull-request-available
>             Fix For: 3.6.0, 3.5.7, 3.7.0
>
>          Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> We have tried to upgrade single node *standalone* ZK server from 3.4.14 to 3.5.6.  There
were no snapshot files, so as suggested in ZOOKEEPER-3056, we have set snapshot.trust.empty
to true. After server startup, when we tried to list the znodes, we found that znodes are
missing.
> Steps to reproduce:
>  # Start a single node ZK 3.4.14 server and create few znodes
>  # Upgrade the server to 3.5.6 with  snapshot.trust.empty=true config
>  # try to list the znodes using zkShell
> Looking into the [source code|https://github.com/apache/zookeeper/blob/release-3.5.6/zookeeper-server/src/main/java/org/apache/zookeeper/server/persistence/FileTxnSnapLog.java#L224],
looks like we are not reading transaction log if there are no snapshot files and snapshot.trust.empty
is set to true.
> ZK 3.5.6 logs:
> {quote}[2019-12-07 12:13:35,007] INFO Created server with tickTime 3000 minSessionTimeout
6000 maxSessionTimeout 60000 datadir /var/lib/zookeeper/version-2 snapdir /var/lib/zookeeper/version-2
>   (org.apache.zookeeper.server.ZooKeeperServer)
>  [2019-12-07 12:13:35,012] INFO Using org.apache.zookeeper.server.NIOServerCnxnFactory
as server connection factory (org.apache.zookeeper.server.ServerCnxnFactory)
>  [2019-12-07 12:13:35,014] INFO Configuring NIO connection handler with 10s sessionless
connection timeout, 1 selector thread(s), 12 worker threads, and 64 kB direct buffers. (org.apache
>  .zookeeper.server.NIOServerCnxnFactory)
>  [2019-12-07 12:13:35,017] INFO binding to port [0.0.0.0/0.0.0.0:2181|http://0.0.0.0/0.0.0.0:2181] (org.apache.zookeeper.server.NIOServerCnxnFactory)
>  [2019-12-07 12:13:35,027] INFO zookeeper.snapshotSizeFactor = 0.33 (org.apache.zookeeper.server.ZKDatabase)
>  [2019-12-07 12:13:35,029] DEBUG Created new input stream /var/lib/zookeeper/version-2/log.1
(org.apache.zookeeper.server.persistence.FileTxnLog)
>  [2019-12-07 12:13:35,031] DEBUG Created new input archive /var/lib/zookeeper/version-2/log.1
(org.apache.zookeeper.server.persistence.FileTxnLog)
>  [2019-12-07 12:13:35,035] DEBUG EOF exception java.io.EOFException: Failed to read /var/lib/zookeeper/version-2/log.1
(org.apache.zookeeper.server.persistence.FileTxnLog)
>  [2019-12-07 12:13:35,035] WARN No snapshot found, but there are log entries. This should
only be allowed during upgrading. (org.apache.zookeeper.server.persistence.FileTxnSnapLog)
>  [2019-12-07 12:13:35,035] INFO Snapshotting: 0x0 to /var/lib/zookeeper/version-2/snapshot.0
(org.apache.zookeeper.server.persistence.FileTxnSnapLog)
>  [2019-12-07 12:13:35,036] INFO Snapshotting: 0x0 to /var/lib/zookeeper/version-2/snapshot.0
(org.apache.zookeeper.server.persistence.FileTxnSnapLog)
>  [2019-12-07 12:13:35,050] INFO Using checkIntervalMs=60000 maxPerMinute=10000 (org.apache.zookeeper.server.ContainerManager)
>  [2019-12-07 12:15:07,137] DEBUG Accepted socket connection from /[127.0.0.1:38888|http://127.0.0.1:38888/] (org.apache.zookeeper.server.NIOServerCnxnFactory)
>  [2019-12-07 12:15:07,144] DEBUG Session establishment request from client /[127.0.0.1:38888|http://127.0.0.1:38888/] client's
lastZxid is 0x0 (org.apache.zookeeper.server.ZooKeeperServer)
>  [2019-12-07 12:15:07,145] DEBUG Adding session 0x100006e15fb0000 (org.apache.zookeeper.server.SessionTrackerImpl)
>  [2019-12-07 12:15:07,148] TRACE SessionTrackerImpl — Adding session 0x100006e15fb0000
30000 (org.apache.zookeeper.server.SessionTrackerImpl)
>  [2019-12-07 12:15:07,149] DEBUG Client attempting to establish new session: session
= 0x100006e15fb0000, zxid = 0x0, timeout = 30000, address = /[127.0.0.1:38888|http://127.0.0.1:38888/] (org.apache.zookeeper.server.ZooKeeperServer)
>  [2019-12-07 12:15:07,155] TRACE :Psessionid:0x100006e15fb0000 type:createSession cxid:0x0
zxid:0xfffffffffffffffe txntype:unknown reqpath:n/a (org.apache.zookeeper.server.PrepRequestProcessor)
>  [2019-12-07 12:15:07,155] TRACE SessionTrackerImpl — Existing session 0x100006e15fb0000
30000 (org.apache.zookeeper.server.SessionTrackerImpl)
>  [2019-12-07 12:15:07,155] INFO Creating new log file: log.1 (org.apache.zookeeper.server.persistence.FileTxnLog)
>  [2019-12-07 12:15:07,170] DEBUG Processing request:: sessionid:0x100006e15fb0000 type:createSession
cxid:0x0 zxid:0x1 txntype:-10 reqpath:n/a (org.apache.zookeeper.server.FinalRequestProcessor)
> {quote}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message