hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wei Tan <w...@us.ibm.com>
Subject Re: memstore timestamp and visible timestamp
Date Fri, 03 Aug 2012 14:21:03 GMT
Hi Lars,

   Appreciate your reply. Actually I read your blog posting and then had 
that question. I am very interested in how you guarantee this:

   Also note that if you use the Region Server assigned TSs then mTS1<mTS2 
implies TS1<=TS2 (the update might happen with the same ms).

  In case you have a pointer explaining this, I would like to read. 
Otherwise I will dig into the code later today. I remember reading 0.92.0 
code and do not find much clue. But I will try again.

 

Best Regards,
Wei

Wei Tan 
Research Staff Member 
IBM T. J. Watson Research Center
19 Skyline Dr, Hawthorne, NY  10532
wtan@us.ibm.com; 914-784-6752



From:   lars hofhansl <lhofhansl@yahoo.com>
To:     "dev@hbase.apache.org" <dev@hbase.apache.org>, 
"hbase-dev@hadoop.apache.org" <hbase-dev@hadoop.apache.org>, 
Date:   08/02/2012 07:35 PM
Subject:        Re: memstore timestamp and visible timestamp



Hi Wei,

you have to distinguish between "visible to other concurrent scanners" and 
"visible to a client".
What's visible to a client is determined by what the a client wants to see 
based on the application visible timestamp (TS).

The visibility to concurrent scanners is controlled by the memstoreTS 
(mTS) to avoid "strange" states sue to parallel updates.
HBase here guards against partially visible "transactions" (i.e. a Put of 
many columns that fails after it applied the changes to some of the 
columns).

The scenario you describe below is indeed desired. Note that a client can 
request seeing the older versions too so the older edit (in terms of TS is 
not lost).
Also note that if you use the Region Server assigned TSs then mTS1<mTS2 
implies TS1<=TS2 (the update might happen with the same ms).

If you do not mind a longer read, I have written about this here: 
http://hadoop-hbase.blogspot.com/2012/03/acid-in-hbase.html

Let me know if that makes any sense.

-- Lars


----- Original Message -----
From: Wei Tan <wtan@us.ibm.com>
To: hbase-dev@hadoop.apache.org
Cc: 
Sent: Thursday, August 2, 2012 3:35 PM
Subject: memstore timestamp and visible timestamp

Hi,

  I have a question regarding the correlation between the visible 
timestamp of a KV (denoted as ts) and its memstore timestamp (aka, the 
write number, denoted as memts). Reading the HRegion.java code it seems 
that these two are independently assigned. Let's assume two concurrent 
put: (k, v1) and (k, v2)


  Suppose somehow memts(k,v1) < memts(k, v2) then (k,v1) will be committed 

and visible before (k,v2). 
If ts(k,v1) < ts(k, v2), then after both KVs commits, (k,v2) becomes the 
latest version.
else, if ts(k,v1) > ts(k, v2), then after a "later"(w.r.t. MVCC) KV 
commits, it immediately become stale and still not visible. --- Is it a 
desirable feature?


  Am I understanding it correctly, that memts(k,v1) < memts(k, v2) does 
not indicate that ts(k,v1) < ts(k, v2), and vice versa? 
PS: let's talk about the hbase region server assigned, not user assigned, 
visible timestamp.

  Thanks,

Wei

Best Regards,
Wei

Wei Tan 
Research Staff Member 
IBM T. J. Watson Research Center
19 Skyline Dr, Hawthorne, NY  10532
wtan@us.ibm.com; 914-784-6752 



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message