hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Re: Timestamp resolution
Date Wed, 11 Jun 2014 21:03:03 GMT
Weirdly enough I find that I have to agree with Andrew. 

First, how do you get time in units smaller than a ms? 
Second clock skew becomes an issue. 
Third, which clock are you using? The client machine? The RS? And then how do you synchronize
each of the RS to be within a ms of each other? 
Correct me if I’m wrong but NTP doesn’t give that close of a sync.  

Sorry, but really, not a good idea. 

If you want this… you can store the temporal data as a column. 

Time really is relative. 

On May 25, 2014, at 12:53 AM, Stack <stack@duboce.net> wrote:

> On Fri, May 23, 2014 at 5:27 PM, lars hofhansl <larsh@apache.org> wrote:
> 
>> We have discussed this in the past. It just came up again during an
>> internal discussion.
>> Currently we simply store a Java timestamp (millisec since epoch), i.e. we
>> have ms resolution.
>> 
>> We do have 8 bytes for the TS, though. Not enough to store nanosecs (that
>> would only cover 2^63/10^9/3600/24/365.24 = 292.279 years), but enough for
>> microseconds (292279 years).
>> Should we just store he TS is microseconds? We could do that right now
>> (and just keep the ms resolution for now - i.e. the us part would always be
>> 0 for now).
>> Existing data must be in ms of course, so we'd grandfather that in, but
>> new tables could store by default in us.
>> 
>> We'd need to make this configurable both the column family level and
>> client level, so clients could still opt to see data in ms.
>> 
>> Comments? Too much to bite off?
>> 
>> -- Lars
>> 
>> 
> I'm a fan.  As Enis cites, HBASE-8927 has good discussion.  No
> configuration I'd say.  Just move to the new regime (though I suppose we
> should let you turn it off).
> 
> I think it was Liu Shaohui (IIRC) who made a suggestion that had us put
> together ms and nanos under a synchronized block stamping the ts on Cells
> (left-shift the currentTimeMillis and fill in the bottom bytes with as much
> of the nanos as fits; i.e. your micros).  Rather than nanos/micros, we
> could use a counter instead if a Cell arrives in the same ms.  Would be
> costly having all ops go via one code block to get 'time' across cores and
> handlers.
> 
> St.Ack


Mime
View raw message