Mailing-List: contact notifications-help@accumulo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@apache.org
Date: Fri, 28 Feb 2014 20:10:32 +0000 (UTC)
From: "ASF subversion and git services (JIRA)" <jira@apache.org>
To: notifications@accumulo.apache.org
Message-ID: <JIRA.12697816.1393540339914.127038.1393618232061@arcas>
In-Reply-To: <JIRA.12697816.1393540339914@arcas>
References: <JIRA.12697816.1393540339914@arcas>
Subject: [jira] [Commented] (ACCUMULO-2422) Backup master can miss acquiring
 lock when primary exits
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/ACCUMULO-2422?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13916305#comment-13916305 ] 

ASF subversion and git services commented on ACCUMULO-2422:
-----------------------------------------------------------

Commit 7eeff02c7cf883765a33575a19d208be30e1e17c in accumulo's branch refs/heads/master from [~bhavanki]
[ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=7eeff02 ]

ACCUMULO-2422 Refine renewal of master lock watcher

The first commit for ACCUMULO-2422 succeeds in renewing the watch on another master's lock
node when needed. This commit refines the solution:

- The renewal was happening even after the master is able to acquire the lock. This led to a
  spurious log error message. This commit skips renewing the watch in that case.
- If the renewal returns a null status, meaning the other master's lock node disappeared, the
  master now immediately tries again to acquire the lock. This matches watch establishment in
  other areas.

A lot of logging at the trace level was added to ZooLock to assist future troubleshooting.


> Backup master can miss acquiring lock when primary exits
> --------------------------------------------------------
>
>                 Key: ACCUMULO-2422
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2422
>             Project: Accumulo
>          Issue Type: Bug
>          Components: fate, master
>    Affects Versions: 1.5.1
>            Reporter: Bill Havanki
>            Assignee: Bill Havanki
>            Priority: Critical
>              Labels: failover, locking
>             Fix For: 1.6.0, 1.5.2
>
>
> While running randomwalk tests with agitation for the 1.5.1 release, I've seen situations where a backup master that is eligible to grab the master lock continues to wait. When this condition arises and the other master restarts, both wait for the lock without success.
> I cannot reproduce the problem reliably, and I think more investigation is needed to see what circumstances could be causing the problem.
> h3. Diagnosis and Work Around
> This failure condition can occur on start up and on backup/active failover of the Master role. If the follow log entry is the final entry on all Master logs you should restart all Master roles, staggering by a few seconds.
> {noformat}
> [master.Master] INFO : trying to get master lock
> {noformat}
> If starting a cluster with multiple Master roles, you should stagger Master role starts by a few seconds.


--
This message was sent by Atlassian JIRA
(v6.1.5#6160)