zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Debraj Manna <subharaj.ma...@gmail.com>
Subject Re: The current epoch, 7, is older than the last zxid, 8589935882
Date Wed, 21 Aug 2019 07:05:45 GMT
With the other two zookeeper servers running I stopped the zookeeper in the
broken node and the deleted all the contents inside
/var/lib/zookeeper/version-2
and started the zookeeper back on the node. It is running fine now and got
all the data from the other servers.

I am getting confused after going through ZOOKEEPER-1653
<https://issues.apache.org/jira/browse/ZOOKEEPER-1653> and ZOOKEEPER-2354
<https://issues.apache.org/jira/browse/ZOOKEEPER-2354> . The issues say it
is fixed in 3.4.6 but exists in 3.5.x. But I am seeing the issue in 3.4.13
also. Can someone let me know if the issue is present in 3.4.13 also?



On Wed, Aug 21, 2019 at 8:54 AM Debraj Manna <subharaj.manna@gmail.com>
wrote:

> Thanks for replying.
>
> What is the recommended way to remove a node and delete all data from it
> and make it start fresh?
>
> On Wed 21 Aug, 2019, 12:58 AM Enrico Olivelli, <eolivelli@gmail.com>
> wrote:
>
>> Hello,
>> Sorry for so late reply.
>> If you have 3 servers you can nuke the broken one and make it start from
>> scratch, it will join the cluster and then recover data from the other
>> servers
>>
>> Try it in a staging env, not in production
>>
>> Enrico
>>
>> Il mar 20 ago 2019, 20:30 Debraj Manna <subharaj.manna@gmail.com> ha
>> scritto:
>>
>> > The same has been asked in stackoverflow
>> > <
>> >
>> https://stackoverflow.com/questions/57574298/zookeeper-error-the-current-epoch-is-older-than-the-last-zxid
>> > >
>> > also. But no response there also.
>> >
>> > Anyone any thoughts on this one?
>> >
>> > On Tue, Aug 20, 2019 at 4:43 PM Debraj Manna <subharaj.manna@gmail.com>
>> > wrote:
>> >
>> > > Posted wrong Jira link. I meant
>> > > https://issues.apache.org/jira/browse/ZOOKEEPER-2354.  Can someone
>> let
>> > me
>> > > know what is the recommended way to recover the node?
>> > >
>> > > support@platform2:/var/lib/zookeeper/version-2$ sudo cat
>> acceptedEpoch
>> > > 8support@platform2:/var/lib/zookeeper/version-2$ sudo cat
>> currentEpoch
>> > > 7support@platform2:/var/lib/zookeeper/version-2$ sudo cat
>> > currentEpoch.tmp
>> > > 8support@platform2
>> > >
>> > > On Tue, Aug 20, 2019 at 3:14 PM Debraj Manna <
>> subharaj.manna@gmail.com>
>> > > wrote:
>> > >
>> > >> Hi
>> > >>
>> > >> I am using a zookeeper ensemble of 3 nodes running 3.4.13. Sometimes
>> > >> after reboot of machine zookeeper is not starting and I am seeing the
>> > below
>> > >> errors in logs.
>> > >>
>> > >> I have seen https://issues.apache.org/jira/browse/ZOOKEEPER-1653 .
>> Can
>> > >> someone let me if this is fixed in 3.4.13 or not as I can see the
>> issue
>> > >> still open? Also can somone suggest what is the recommended way to
>> > recover
>> > >> the set-up ?
>> > >>
>> > >> 2019-08-19 04:18:36,906 [myid:2] - ERROR [main:QuorumPeer@692] -
>> Unable
>> > >> to load database on disk
>> > >> java.io.IOException: The current epoch, 7, is older than the last
>> zxid,
>> > >> 34359738370
>> > >> at
>> > >>
>> >
>> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:674)
>> > >> at
>> > >>
>> org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:635)
>> > >> at
>> > >>
>> >
>> org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:170)
>> > >> at
>> > >>
>> >
>> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:114)
>> > >> at
>> > >>
>> >
>> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)
>> > >> 2019-08-19 04:18:36,908 [myid:2] - ERROR [main:QuorumPeerMain@92] -
>> > >> Unexpected exception, exiting abnormally
>> > >> java.lang.RuntimeException: Unable to run quorum server
>> > >> at
>> > >>
>> >
>> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:693)
>> > >> at
>> > >>
>> org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:635)
>> > >> at
>> > >>
>> >
>> org.apache.zookeeper.server.quorum.QuorumPeerMain.runFromConfig(QuorumPeerMain.java:170)
>> > >> at
>> > >>
>> >
>> org.apache.zookeeper.server.quorum.QuorumPeerMain.initializeAndRun(QuorumPeerMain.java:114)
>> > >> at
>> > >>
>> >
>> org.apache.zookeeper.server.quorum.QuorumPeerMain.main(QuorumPeerMain.java:81)
>> > >> Caused by: java.io.IOException: The current epoch, 7, is older than
>> the
>> > >> last zxid, 34359738370
>> > >> at
>> > >>
>> >
>> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:674)
>> > >> ... 4 more----
>> > >>
>> > >>
>> > >>
>> >
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message