Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Fri, 22 Apr 2016 12:10:13 +0000 (UTC)
From: "Robert Stupp (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12957853.1460407138000.7729.1461327013028@Atlassian.JIRA>
In-Reply-To: <JIRA.12957853.1460407138000@Atlassian.JIRA>
References: <JIRA.12957853.1460407138000@Atlassian.JIRA>
 <JIRA.12957853.1460407138922@arcas>
Subject: [jira] [Commented] (CASSANDRA-11547) Add background thread to check
 for clock drift
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-11547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15253812#comment-15253812 ] 

Robert Stupp commented on CASSANDRA-11547:
------------------------------------------

I'm in favor of having this patch. Bad things happen in the wild and I agree to the argument that having something that gives an operator an indication that the wall clock somehow went too fast or too slow can definitely explain things that would otherwise be considered as "magic" or "strange behavior".
Making it a bit more configurable to the points mentioned by Sylvain would be a good thing.

> Add background thread to check for clock drift
> ----------------------------------------------
>
>                 Key: CASSANDRA-11547
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11547
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Jason Brown
>            Assignee: Jason Brown
>            Priority: Minor
>              Labels: clocks, time
>
> The system clock has the potential to drift while a system is running. As a simple way to check if this occurs, we can run a background thread that wakes up every n seconds, reads the system clock, and checks to see if, indeed, n seconds have passed. 
> * If the clock's current time is less than the last recorded time (captured n seconds in the past), we know the clock has jumped backward.
> * If n seconds have not elapsed, we know the system clock is running slow or has moved backward (by a value less than n)
> * If (n + a small offset) seconds have elapsed, we can assume we are within an acceptable window of clock movement. Reasons for including an offset are the clock checking thread might not have been scheduled on time, or garbage collection, and so on.
> * If the clock is greater than (n + a small offset) seconds, we can assume the clock jumped forward.
> In the unhappy cases, we can write a message to the log and increment some metric that the user's monitoring systems can trigger/alert on.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)