hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-11778) Scale timestamps by 1000
Date Wed, 20 Aug 2014 07:10:26 GMT

    [ https://issues.apache.org/jira/browse/HBASE-11778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14103536#comment-14103536

Lars Hofhansl commented on HBASE-11778:

I doubt multiplying by 1000 vs. shifting 16 bits will have any impact on the overall path.
Would need to explain our resolution: 1/65536000s. Sounds bad. :)

But shifting is fine, of course.

What I am concerned about is:
* making easy sense out of the new numbers (that can be fixed by the right APIs of course)
* allowing enough of a time range. ns would give is 292 years, us 292000 years. If we shift
by 16 bits we'd have 4462 years. If we shifted by 20 bits we could get ns and 278 years.

Anyway... Shall we continue on HBASE-8927?

> Scale timestamps by 1000
> ------------------------
>                 Key: HBASE-11778
>                 URL: https://issues.apache.org/jira/browse/HBASE-11778
>             Project: HBase
>          Issue Type: Brainstorming
>            Reporter: Lars Hofhansl
> The KV timestamps are used for various reasons:
> # ordering of KVs
> # resolving conflicts
> # enforce TTL
> Currently we assume that the timestamps have a resolution of 1ms, and because of that
we made the resolution at which we can determine time identical to the resolution at which
we can store time.
> I think it is time to disentangle the two... At least allow a higher resolution of time
to be stored. That way we could have a centralized transaction oracle that produces ids that
relate to wall clock time, and at the same time allow producing more than 1000/s.
> The simplest way is to just store time in us (microseconds). I.e. we'd still collect
time in ms by default and just multiply that with 1000 before we store it. With 8 bytes that
still gives us a range of 292471 years.
> We'd have grandfather in old data. Could write a metadata entry into each HFile declaring
what the TS resolution is if it is different from ms.
> Not sure, yet, how this would relate to using the TS for things like seqIds.
> Let's do some brainstorming. 

This message was sent by Atlassian JIRA

View raw message