hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: Timestamp resolution
Date Sat, 24 May 2014 01:21:21 GMT
Thank you very much for the explanation - makes sense!

And I would love to put the heads together with you at the end of 297th year
to contemplate on why we thought that 1m trans/sec was sufficient ;) This is
certainly something to look forward to!
 
Cos

On Fri, May 23, 2014 at 06:16PM, lars hofhansl wrote:
> The specific discussion here was a transaction engine doing snapshot
> isolation using the HBase timestamps, but still be close to wall clock time
> as much as possible.
> In that scenario, with ms resolution you can only do 1000 transactions/sec,
> and so you need to turn the timestamp into something that is not wall clock
> time as HBase understands it (and hence TTL, etc, will no longer work, as
> well as any other tools you've written that use the HBase timestamp).
> 1m transactions/sec are good enough (for now, I envision in a few years
> we'll be sitting here wondering how we could ever think that 1m
> transaction/sec are sufficient) :)
> 
> -- Lars
> 
> 
> 
> ________________________________
>  From: Konstantin Boudnik <cos@apache.org>
> To: dev@hbase.apache.org; lars hofhansl <larsh@apache.org> 
> Sent: Friday, May 23, 2014 5:58 PM
> Subject: Re: Timestamp resolution
>  
> 
> What's the purpose of nanos accuracy in the TS? I am trying to think of one,
> but I don't know much about real production use cases.
> 
> Cos
> 
> P.S. Are you saying that a real concern is how usable Hbase will be in
> nearly 300 years from now? ;) Or I misread you? 
> 
> 
> On Fri, May 23, 2014 at 05:27PM, lars hofhansl wrote:
> > We have discussed this in the past. It just came up again during an internal discussion.
> > Currently we simply store a Java timestamp (millisec since epoch), i.e. we have
ms resolution.
> > 
> > We do have 8 bytes for the TS, though. Not enough to store nanosecs (that
> > would only cover 2^63/10^9/3600/24/365.24 = 292.279 years), but enough for
> > microseconds (292279 years).
> > Should we just store he TS is microseconds? We could do that right now (and
> > just keep the ms resolution for now - i.e. the us part would always be 0 for
> > now).
> > Existing data must be in ms of course, so we'd grandfather that in, but new
> > tables could store by default in us.
> > 
> > We'd need to make this configurable both the column family level and client
> > level, so clients could still opt to see data in ms.
> > 
> > Comments? Too much to bite off?
> > 
> > -- Lars
> > 

Mime
View raw message