hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lars hofhansl <lhofha...@yahoo.com>
Subject Re: memstore timestamp and visible timestamp
Date Thu, 02 Aug 2012 23:33:31 GMT
Hi Wei,

you have to distinguish between "visible to other concurrent scanners" and "visible to a client".
What's visible to a client is determined by what the a client wants to see based on the application
visible timestamp (TS).

The visibility to concurrent scanners is controlled by the memstoreTS (mTS) to avoid "strange"
states sue to parallel updates.
HBase here guards against partially visible "transactions" (i.e. a Put of many columns that
fails after it applied the changes to some of the columns).

The scenario you describe below is indeed desired. Note that a client can request seeing the
older versions too so the older edit (in terms of TS is not lost).
Also note that if you use the Region Server assigned TSs then mTS1<mTS2 implies TS1<=TS2
(the update might happen with the same ms).

If you do not mind a longer read, I have written about this here: http://hadoop-hbase.blogspot.com/2012/03/acid-in-hbase.html

Let me know if that makes any sense.

-- Lars

----- Original Message -----
From: Wei Tan <wtan@us.ibm.com>
To: hbase-dev@hadoop.apache.org
Sent: Thursday, August 2, 2012 3:35 PM
Subject: memstore timestamp and visible timestamp


  I have a question regarding the correlation between the visible 
timestamp of a KV (denoted as ts) and its memstore timestamp (aka, the 
write number, denoted as memts). Reading the HRegion.java code it seems 
that these two are independently assigned. Let's assume two concurrent 
put: (k, v1) and (k, v2)

  Suppose somehow memts(k,v1) < memts(k, v2) then (k,v1) will be committed 
and visible before (k,v2). 
If ts(k,v1) < ts(k, v2), then after both KVs commits, (k,v2) becomes the 
latest version.
else, if ts(k,v1) > ts(k, v2), then after a "later"(w.r.t. MVCC) KV 
commits, it immediately become stale and still not visible. --- Is it a 
desirable feature?

  Am I understanding it correctly, that memts(k,v1) < memts(k, v2) does 
not indicate that ts(k,v1) < ts(k, v2), and vice versa? 
PS: let's talk about the hbase region server assigned, not user assigned, 
visible timestamp.



Best Regards,

Wei Tan 
Research Staff Member 
IBM T. J. Watson Research Center
19 Skyline Dr, Hawthorne, NY  10532
wtan@us.ibm.com; 914-784-6752 

View raw message