hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: Nondeterministic outcome based on cell TTL and major compaction event order
Date Fri, 17 Apr 2015 21:37:14 GMT
The way TTLs work today is they define the interval of time a cell
exists - exactly as that. There is no tombstone laid like a normal
delete. Once the TTL elapses the cell just ceases to exist to normal
scanners. The interaction of expired cells, multiple versions, minimum
versions, raw scanners, etc. can be confusing. We can absolutely
revisit this.

A cell with an expired TTL could be treated as the combination of
tombstone and the most recent value it lays over. This is not how the
implementation works today, but could be changed for an upcoming major
version like 2.0 if there's consensus to do it.


> On Apr 10, 2015, at 7:26 AM, Cosmin Lehene <clehene@adobe.com> wrote:
>
> I've been initially puzzled by this, although I realize how it's likely as designed.
>
>
> The cell TTL expiration and compactions events can lead to either some (the older) data
left or no data at all for a particular  (row, family, qualifier, ts) coordinate.
>
>
>
> Write (r1, f1, q1, v1, 1)
>
> Write (r1, f1, q1, v1, 2) - TTL=1 minute
>
>
> Scenario 1:
>
>
> If a major compaction happens within a minute
>
>
> it will remove (r1, f1, q1, v1, 1)
>
> then after a minute (r1, f1, q1, v1, 2) will expire
>
> no data left
>
>
> Scenario 2:
>
>
> A minute passes
>
> (r1, f1, q1, v1, 2) expires
>
> Compaction runs..
>
> (r1, f1, q1, v1, 1) remains
>
>
>
> This seems, by and large expected behavior, but it still seems "uncomfortable" that the
(overall) outcome is not decided by me, but by a chance of event ordering.
>
>
> I wonder we'd want this to behave differently (perhaps it has been discussed already),
but if not, it's worth a more detailed documentation in the book.
>
>
> What do you think?
>
>
> Cosmin
>
>
>
>

-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet
Hein (via Tom White)

Mime
View raw message