activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Tully (JIRA)" <>
Subject [jira] [Commented] (AMQ-4122) Lease Database Locker failover broken
Date Fri, 22 Feb 2013 19:50:14 GMT


Gary Tully commented on AMQ-4122:

@SouNayi - thanks for the slq log.
>From looking at, it
looks like a configuration problem.
node-h03-ap21 is obtaining a 5s lease that it renews every 10s. So there is a s period when
the lease is available to others.

It needs to obtain a 10 second lease and update it every 5 seconds. So that a second (slave)
broker always sees time > now when it attempts an update as part of an acquire.

You need:
<jdbcPersistenceAdapter ... lockKeepAlivePeriod="5000">
     <lease-database-locker lockAcquireSleepInterval="10000"/>
lockAcquireSleepInterval is the lease duration, lockKeepAlivePeriod is the lease renew period.
On a renew, the lease is extended by the lockAcquireSleepInterval (lease duration). So a master
is always (lockAcquireSleepInterval - lockKeepAlivePeriod) ahead with its lease.
In short, ensure: lockAcquireSleepInterval > lockKeepAlivePeriod.

Can you verify this.
I think it may also makes sense to add lease related attributes to this locker. leaseDuration,
leaseRenewPeriod so that it is a little more intuitive and obvious that the leaseDuration
> leaseRenewPeriod

> Lease Database Locker failover broken
> -------------------------------------
>                 Key: AMQ-4122
>                 URL:
>             Project: ActiveMQ
>          Issue Type: Bug
>    Affects Versions: 5.7.0
>         Environment: Java 7u9, SUSE 11, Mysql
>            Reporter: st.h
>            Assignee: Gary Tully
>             Fix For: 5.8.0
>         Attachments: activemq-kyle.xml, activemq.xml, activemq.xml, AMQ4122.patch, mysql.log
> We are using ActiveMQ 5.7.0 together with a mysql database and could not observe correct
failover behavior with lease database locker.
> It seems that there is a race condition, which prevents the correct failover procedure.
> We noticed that when starting up two instances, both instance are becoming master.
> We did several test, including the following and could not observe intended functionality:
> - shutdown all instances
> - manipulate database lock that one node has lock and set expiry time in distance future
> - start up both instances. both instances are unable to acquire lock, as the lock hasn't
expired, which should be correct behavior.
> - update the expiry time in database, so that the lock is expired.
> - first instance notices expired lock and becomes master
> - when second instance checks for lock, it also updates the database and becomes master.
> To my understanding the second instance should not be able to update the lock, as it
is held by the first instance and should not be able to become master.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message