hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Appy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14070) Hybrid Logical Clocks for HBase
Date Thu, 20 Jul 2017 00:39:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16094013#comment-16094013

Appy commented on HBASE-14070:

While reviewing one of the patches, a question popped in my mind:
We are passing master clock to RS in open/close region request and RS clock to master in the
responses. And they both update their own time on receiving these request/response.
(x= don't care)
Say RS hlc time is (10,x) and master time is (20,x). Master sends request to RS which then
updates its own time to (20,x).
While RS's physical java clock (which is backing up physical component of hlc clock) will
still take 10 sec to catch up, we'll keep incrementing logical component. That means, in worst
case, our logical clock window should be big enough to support all the events that can happen
in max skew time.
The problem is, that doesn't seem to be the case. Our logical window is 1M events (20bits)
and max skew time is 30 sec, that results in 33k  max write qps, which is quite low. We can
easily see 150k update qps per beefy server with 1k values.

Even 22 bits won't be enough. We'll need minimum of 23 bits and 20 sec max skew time to get
to ~420k events per second (worst case).

> Hybrid Logical Clocks for HBase
> -------------------------------
>                 Key: HBASE-14070
>                 URL: https://issues.apache.org/jira/browse/HBASE-14070
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Enis Soztutar
>            Assignee: Amit Patel
>         Attachments: HBASE-14070.master.001.patch, HybridLogicalClocksforHBaseandPhoenix.docx,
> HBase and Phoenix uses systems physical clock (PT) to give timestamps to events (read
and writes). This works mostly when the system clock is strictly monotonically increasing
and there is no cross-dependency between servers clocks. However we know that leap seconds,
general clock skew and clock drift are in fact real. 
> This jira proposes using Hybrid Logical Clocks (HLC) as an implementation of hybrid physical
clock + a logical clock. HLC is best of both worlds where it keeps causality relationship
similar to logical clocks, but still is compatible with NTP based physical system clock. HLC
can be represented in 64bits. 
> A design document is attached and also can be found here: 
> https://docs.google.com/document/d/1LL2GAodiYi0waBz5ODGL4LDT4e_bXy8P9h6kWC05Bhw/edit#

This message was sent by Atlassian JIRA

View raw message