cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Paulo Motta (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-14071) Materialized view on table with TTL issue
Date Fri, 01 Dec 2017 21:21:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-14071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16275007#comment-16275007
] 

Paulo Motta edited comment on CASSANDRA-14071 at 12/1/17 9:20 PM:
------------------------------------------------------------------

bq. In 11500, an expired livenessInfo concept is introduced to MV, in order to mark entire
view row as dead without purging row cells as tombstone. When table has default ttl, the MV
row to be removed has a bigger localDeletionTime than the generated expired livenessInfo's.
(see LivenessInfo.supersedes())

Good investigation! On #11500 we focused on the case were TTLs are mixed with ordinary writes
but missed the overwrite with ttl case.

bq. The fix will be: when livenessInfo timestamp ties, MV expired livenessInfo (by checking
TTL==MAX) will supersedes another. I will check again if there is anything left for MV reconciliation.

The approach looks good to me but even though this looks safe I think we should restrict this
new resolution rule only to pk liveness info of views to ensure no other code path can be
affected by this since this is a pretty critical path. What do you think? 

Also, besides the case with default ttl, this also affects overwrites with ttl right? Can
you add a test with this case as well?

Thanks!


was (Author: pauloricardomg):
bq. In 11500, an expired livenessInfo concept is introduced to MV, in order to mark entire
view row as dead without purging row cells as tombstone. When table has default ttl, the MV
row to be removed has a bigger localDeletionTime than the generated expired livenessInfo's.
(see LivenessInfo.supersedes())

Good investigation! On #11500 we focused on the case were TTLs are mixed with ordinary writes
but missed the overwrite with ttl case.

bq. The fix will be: when livenessInfo timestamp ties, MV expired livenessInfo (by checking
TTL==MAX) will supersedes another. I will check again if there is anything left for MV reconciliation.

The approach looks good to me but even though this looks safe I think we should restrict this
new resolution rule only to pk liveness info of views to ensure no other code path will can
be affected by this since this is a pretty critical path. What do you think? 

Also, besides the case with default ttl, this also affects overwrites with ttl right? Can
you add a test with this case as well?

Thanks!

> Materialized view on table with TTL issue
> -----------------------------------------
>
>                 Key: CASSANDRA-14071
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-14071
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Coordination, Materialized Views
>         Environment: Cassandra 3
>            Reporter: Silviu Butnariu
>            Assignee: ZhaoYang
>              Labels: correctness
>
> Materialized views that cluster by a column that is not part of table's PK and are created
from tables that have *default_time_to_live* seems to malfunction.
> Having this table
> {code:java}
> CREATE TABLE sbutnariu.test_bug (
>     field1 smallint,
>     field2 smallint,
>     date timestamp,
>     PRIMARY KEY ((field1), field2)
> ) WITH default_time_to_live = 1000;
> {code}
> and the materialized view
> {code:java}
> CREATE MATERIALIZED VIEW sbutnariu.test_bug_by_date AS SELECT * FROM sbutnariu.test_bug
WHERE field1 IS NOT NULL AND field2 IS NOT NULL AND date IS NOT NULL PRIMARY KEY ((field1),
date, field2) WITH CLUSTERING ORDER BY (date desc, field2 asc);
> {code}
> After inserting 3 rows with same PK (should upsert), the materialized view will have
3 rows.
> {code:java}
> insert into sbutnariu.test_bug(field1, field2, date) values (1, 2, toTimestamp(now()));
> insert into sbutnariu.test_bug(field1, field2, date) values (1, 2, toTimestamp(now()));
> insert into sbutnariu.test_bug(field1, field2, date) values (1, 2, toTimestamp(now()));
> select * from sbutnariu.test_bug; /*1 row*/
> select * from sbutnariu.test_bug_by_date;/*3 rows*/
> {code}
> If I remove the ttl and try again, it works as expected:
> {code:java}
> truncate sbutnariu.test_bug;
> alter table sbutnariu.test_bug with default_time_to_live = 0;
> select * from sbutnariu.test_bug; /*1 row*/
> select * from sbutnariu.test_bug_by_date;/*1 row*/
> {code}
> I've tested on versions 3.0.14 and 3.0.15. The bug was introduced in 3.0.15, as in 3.0.14
it works as expected.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message