curator-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cameron McKenzie <mckenzie....@gmail.com>
Subject Re: Zookeeper Leadership election with curator
Date Fri, 16 Jan 2015 04:06:13 GMT
The InterProcessMutex is essentially a distributed lock. In each of your
clients you just create an InterProcessMutex instance referring to the same
ZK path

CuratorFramework curator = ...
String lockPath = "/lockPath";

InterProcessMutex mutex = new InterProcessMutex(curator, lockPath);
try {
    //This will block until the mutex is acquire.
    mutex.acquire();

    //Do your critical stuff
} finally {
    //Release the mutex
    mutex.release();
}




On Fri, Jan 16, 2015 at 12:05 PM, Check Peck <comptechgeeky@gmail.com>
wrote:

> What is the benefit I will have with InterProcessMutex? I have never use
> this before. Is there any example which I can take a look?
>
> On Thu, Jan 15, 2015 at 4:07 PM, Cameron McKenzie <mckenzie.cam@gmail.com>
> wrote:
>
>> Yes, good point.
>>
>> On Fri, Jan 16, 2015 at 10:57 AM, Jordan Zimmerman <
>> jordan@jordanzimmerman.com> wrote:
>>
>>> But if you go through that much trouble you might as well just use
>>> InterProcessMutex.
>>>
>>> -JZ
>>>
>>>
>>>
>>> On January 15, 2015 at 6:54:30 PM, Cameron McKenzie (
>>> mckenzie.cam@gmail.com) wrote:
>>>
>>>   If you use the callback as Jordan suggested, then you can just block
>>> on a latch, and then get this latch to be updated by the callback when it
>>> is notified that it's leader. Then you won't have to wait for some fixed
>>> amount of time, it will just block until it's leader.
>>>
>>>
>>> On Fri, Jan 16, 2015 at 10:48 AM, Check Peck <comptechgeeky@gmail.com>
>>> wrote:
>>>
>>>> As of now, I am adding a sleep of 1 minute and it works fine before I
>>>> check whether I am a leader or not.
>>>>
>>>> On Thu, Jan 15, 2015 at 3:43 PM, Check Peck <comptechgeeky@gmail.com>
>>>> wrote:
>>>>
>>>>>  Yes that makes sense that 1 client will be leader. But suppose I
>>>>> have 5 background threads running in 5 different machines and each
>>>>> background thread is in different classes and I want to make sure that
>>>>> those 5 background threads are running from a single machine who is the
>>>>> leader so that's why we are doing that check whether I am the leader
then
>>>>> do this.
>>>>>
>>>>> Will this work if we add a Listener to leaderlatch?
>>>>>
>>>>> On Thu, Jan 15, 2015 at 3:30 PM, Jordan Zimmerman <
>>>>> jordan@jordanzimmerman.com> wrote:
>>>>>
>>>>>>  The problem with that is you have to wait until you are the leader.
>>>>>> The benefit of LeaderSelector is that it has a callback that is invoked
>>>>>> when you are the leader. Remember, you are in a multithreaded, multi
client
>>>>>> environment. Only 1 client is going to be the leader. Maybe what
you are
>>>>>> after is an InterProcessMutex?
>>>>>>
>>>>>>  -JZ
>>>>>>
>>>>>>
>>>>>>
>>>>>> On January 15, 2015 at 6:24:06 PM, Check Peck (
>>>>>> comptechgeeky@gmail.com) wrote:
>>>>>>
>>>>>>   I saw the leader election code here -
>>>>>> https://git-wip-us.apache.org/repos/asf?p=curator.git;a=tree;f=curator-examples/src/main/java/leader;h=73b547eadb98995c0ccbd06a5b76d0741ffef263;hb=HEAD
>>>>>>
>>>>>> But didn't understand few things - How can I call something like
this
>>>>>> in my code  - I want to do stuff in my code by just checking whether
I am
>>>>>> the leader or not. And from the LeaderSelectorExample, I am not able
to
>>>>>> understand this.
>>>>>>
>>>>>> if (zookClient.isLeader()) {
>>>>>>
>>>>>> // then do stuff
>>>>>>
>>>>>> }
>>>>>>
>>>>>>
>>>>>> On Thu, Jan 15, 2015 at 3:17 PM, Jordan Zimmerman <
>>>>>> jordan@jordanzimmerman.com> wrote:
>>>>>>
>>>>>>>  Oh - maybe I read the code too fast. I guess it doesn’t make
more
>>>>>>> than one…
>>>>>>>
>>>>>>>  There’s an example of LeaderSelector here:
>>>>>>> http://curator.apache.org/curator-examples/index.html
>>>>>>>
>>>>>>>
>>>>>>> On January 15, 2015 at 6:00:03 PM, Check Peck (
>>>>>>> comptechgeeky@gmail.com) wrote:
>>>>>>>
>>>>>>>    Thanks Jordan for the input.
>>>>>>>
>>>>>>> * Where do you see that I am creating new Curator instance for
each
>>>>>>> Leader? I guess I am using one Curator instance for my application?
>>>>>>> * Ok, I have commented out this line in ZookeeperClient class
>>>>>>> * That's a good point, how would I use the listener or the
>>>>>>> leaderselector in my code base? If possible, can you provide
an example?
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Jan 15, 2015 at 2:38 PM, Jordan Zimmerman <
>>>>>>> jordan@jordanzimmerman.com> wrote:
>>>>>>>
>>>>>>>>  A few things:
>>>>>>>>
>>>>>>>>  * You don’t need to create a new Curator instance for
each Leader.
>>>>>>>> One Curator instance can be used for an application.
>>>>>>>>  * Calling
>>>>>>>> client.getZookeeperClient().blockUntilConnectedOrTimedOut();
is unnecessary
>>>>>>>>  * Calling isLeader() immediately after starting the latch
may not
>>>>>>>> work. It takes time for the leader to get selected. LeaderLatch
has a
>>>>>>>> listener if you want to get notified. Alternatively, you
can use
>>>>>>>> LeaderSelector.
>>>>>>>>
>>>>>>>>  Hope this helps.
>>>>>>>>
>>>>>>>>  -Jordan
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On January 15, 2015 at 5:25:26 PM, Check Peck (
>>>>>>>> comptechgeeky@gmail.com) wrote:
>>>>>>>>
>>>>>>>>  I am using Zookeeper for the leadership election process.
I have
>>>>>>>> three nodes zookeeper ensemble.
>>>>>>>>
>>>>>>>> We have a node like this "/test/msn" on which we are doing
the
>>>>>>>> leadership election and it works fine and we have application
servers
>>>>>>>> running it in production and using "/test/msn" node for leadership.
And
>>>>>>>> this program is always running.
>>>>>>>>
>>>>>>>> Now we created another node like this "/testpnl" in the same
>>>>>>>> zookeeper servers. And we have another set of machines which
is trying to
>>>>>>>> do the leadership election on that node and whenever I run
my program in
>>>>>>>> production from one servers, it always returns back that
the node is not
>>>>>>>> leader? And I can see only one node inside "/testpnl" and
it is the same
>>>>>>>> node from which I am running my program?
>>>>>>>>
>>>>>>>> I am using Curator 2.7.0 and the zookeeper version is 3.4.5
Is
>>>>>>>> there anything wrong with my setup or zookeeper has some
bug which I am not
>>>>>>>> aware?
>>>>>>>>
>>>>>>>> Is anything
>>>>>>>>
>>>>>>>>     public class ZookeeperLeaderTester {
>>>>>>>>
>>>>>>>>         private ZookeeperClient zookClient;
>>>>>>>>
>>>>>>>>         private static final String ZOOK_PRODUCTION =
>>>>>>>> "host1:2181,host2:2181,host3:2181";
>>>>>>>>         private static final String LEADERSHIP_NODE = "/testpnl";
>>>>>>>>
>>>>>>>>         private static class Holder {
>>>>>>>>             static final ZookeeperLeaderTester INSTANCE =
new
>>>>>>>> ZookeeperLeaderTester();
>>>>>>>>         }
>>>>>>>>
>>>>>>>>         public static ZookeeperLeaderTester getInstance()
{
>>>>>>>>             return Holder.INSTANCE;
>>>>>>>>         }
>>>>>>>>
>>>>>>>>         private ZookeeperLeaderTester() {
>>>>>>>>             try {
>>>>>>>>                 String hostname = getHostName();
>>>>>>>>
>>>>>>>>                 zookClient = new ZookeeperClient(nodes,
>>>>>>>> LEADERSHIP_NODE, hostname);
>>>>>>>>                 zookClient.start();
>>>>>>>>
>>>>>>>>             } catch (Exception ex) {
>>>>>>>>                 System.out.println (ex);
>>>>>>>>                 System.exit(1);
>>>>>>>>             }
>>>>>>>>         }
>>>>>>>>
>>>>>>>>         public ZookeeperClient getZookClient() {
>>>>>>>>             return zookClient;
>>>>>>>>         }
>>>>>>>>     }
>>>>>>>>
>>>>>>>> And here is my ZookeeperClient code -
>>>>>>>>
>>>>>>>>     public class ZookeeperClient {
>>>>>>>>
>>>>>>>>         private CuratorFramework client;
>>>>>>>>         private String latchPath;
>>>>>>>>         private String id;
>>>>>>>>         private LeaderLatch leaderLatch;
>>>>>>>>
>>>>>>>>         public ZookeeperClient(String connString, String
latchPath,
>>>>>>>> String id) {
>>>>>>>>             client = CuratorFrameworkFactory.newClient(connString,
>>>>>>>> new ExponentialBackoffRetry(1000, Integer.MAX_VALUE));
>>>>>>>>             this.id = id;
>>>>>>>>             this.latchPath = latchPath;
>>>>>>>>         }
>>>>>>>>
>>>>>>>>         public void start() throws Exception {
>>>>>>>>             client.start();
>>>>>>>>
>>>>>>>> client.getZookeeperClient().blockUntilConnectedOrTimedOut();
>>>>>>>>             leaderLatch = new LeaderLatch(client, latchPath,
id);
>>>>>>>>             leaderLatch.start();
>>>>>>>>         }
>>>>>>>>
>>>>>>>>         public boolean isLeader() {
>>>>>>>>             return leaderLatch.hasLeadership();
>>>>>>>>         }
>>>>>>>>
>>>>>>>>         public Participant currentLeader() throws Exception
{
>>>>>>>>             return leaderLatch.getLeader();
>>>>>>>>         }
>>>>>>>>
>>>>>>>>         public void close() throws IOException {
>>>>>>>>             leaderLatch.close();
>>>>>>>>             client.close();
>>>>>>>>         }
>>>>>>>>
>>>>>>>>         public CuratorFramework getClient() {
>>>>>>>>             return client;
>>>>>>>>         }
>>>>>>>>
>>>>>>>>         public String getLatchPath() {
>>>>>>>>             return latchPath;
>>>>>>>>         }
>>>>>>>>
>>>>>>>>         public String getId() {
>>>>>>>>             return id;
>>>>>>>>         }
>>>>>>>>
>>>>>>>>         public LeaderLatch getLeaderLatch() {
>>>>>>>>             return leaderLatch;
>>>>>>>>         }
>>>>>>>>     }
>>>>>>>>
>>>>>>>> And this is the way I am using it -
>>>>>>>>
>>>>>>>>     ZookeeperLeaderTester zookClient =
>>>>>>>> ZookeeperLeaderTester.getInstance().getZookClient();
>>>>>>>>     if (zookClient.isLeader()) {
>>>>>>>>         // then do stuff
>>>>>>>>     }
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>

Mime
View raw message