zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Junqueira <...@apache.org>
Subject Re: How to handle zookeeper data inconsistency
Date Thu, 21 Jan 2016 13:40:42 GMT
Hi Mohammad,

A delete operation only needs to reach a quorum to complete and A B form a quorum in your
3-server ensemble. If the delete operation never gets propagated to C and other write operations
that have been ordered later complete on C, then you have an issue. If C simply stops receiving
updates, then you have a problem with your C server and it could be a problem with ZK or just
the environment.

If there has been write operations ordered after the delete and server C has seen those but
not the delete, then I'd recommend that you have a look the txn logs with the log formatter.

> shall I check exists from leader only? but even leader can have some node undeleted in
the above scenario

There is no such a requirement, but you need to be aware that server C could definitely make
an update visible later compared to other servers. ZooKeeper doesn't guarantee that updates
are visible to all clients as soon as they are acknowledged.

I'd also search for jiras, especially if you're deleting an ephemeral. 

-Flavio

> On 21 Jan 2016, at 13:24, Mohammad arshad <mohammad.arshad@huawei.com> wrote:
> 
> Hi All,
> I came across a scenario where zookeeper was left in inconsistent state(but that is valid
as per the zookeeper theory) and because of that dependent application's behaved wrongly
> The scenario is as follow
> 
> 1) I have three server Zookeeper cluster, let's say servers are A, B and C. B is the
leader
> 2) In one successful delete operation, a znode znode1 was deleted from A and B but somehow
not deleted from C. The reason for not deleted from C can be either proposal or commit failed.
> 3) Now for application, which is connected to C, ZooKeeper.exists  returns the znod1
and that is why application enters into node exists flow which is wrong
> 
> shall I check exists from leader only? but even leader can have some node undeleted in
the above scenario
> Any guideline to handle the above said valid data inconsistency ??
> 
> Any suggestion/help is highly appreciated.
> 
> Best Regards
> Mohammad Arshad
> HUAWEI TECHNOLOGIES CO.LTD.
> Huawei Tecnologies India Pvt. Ltd.
> Near EPIP Industrial Area, Kundalahalli Village
> Whitefield, Bangalore-560066
> www.huawei.com<http://www.huawei.com/>
> -----------------------------------------------------------------------------------------------------------------
> This e-mail and its attachments contain confidential information from HUAWEI, which
> is intended only for the person or entity whose address is listed above. Any use of the
> information contained herein in any way (including, but not limited to, total or partial
> disclosure, reproduction, or dissemination) by persons other than the intended
> recipient(s) is prohibited. If you receive this e-mail in error, please notify the sender
by
> phone or email immediately and delete it!
> 


Mime
View raw message