hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HBASE-11778) Scale timestamps by 1000
Date Tue, 19 Aug 2014 20:13:19 GMT
Lars Hofhansl created HBASE-11778:

             Summary: Scale timestamps by 1000
                 Key: HBASE-11778
                 URL: https://issues.apache.org/jira/browse/HBASE-11778
             Project: HBase
          Issue Type: Brainstorming
            Reporter: Lars Hofhansl

The KV timestamps are used for various reasons:
# ordering of KVs
# resolving conflicts
# enforce TTL

Currently we assume that the timestamps have a resolution of 1ms, and because of that we made
the resolution at which we can determine time identical to the resolution at which we can
store time.
I think it is time to disentangle the two... At least allow a higher resolution of time to
be stored. That way we could have a centralized transaction oracle that produces ids that
relate to wall clock time, and at the same time allow producing more than 1000/s.

The simplest way is to just store time in us (microseconds). I.e. we'd still collect time
in ms by default and just multiply that with 1000 before we store it. With 8 bytes that still
gives us a range of 292471 years.

We'd have grandfather in old data. Could write a metadata entry into each HFile declaring
what the TS resolution is if it is different from ms.

Not sure, yet, how this would relate to using the TS for things like seqIds.

Let's do some brainstorming. 

This message was sent by Atlassian JIRA

View raw message