cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Carl Yeksigian (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-6477) Materialized Views (was: Global Indexes)
Date Fri, 05 Jun 2015 20:56:05 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14575204#comment-14575204
] 

Carl Yeksigian commented on CASSANDRA-6477:
-------------------------------------------

Update on this ticket:

The consistency issue was caused by not using the batch log for the materialized view replica
mutations. This meant that we weren't guaranteed that the materialized view would get all
of the updates, including tombstones, which is why there were additional entries in the MV.

Adding this fixed the consistency issue, but lengthened the time to execute a materialized
view mutation, which exacerbated another issue: lock contention for the partition. Since we
need to make sure that the MV and the base table are not updated until we have finished figuring
out the new mutations; trying to acquire that lock also backs up the mutation stage and causes
the batchlog mutations to fail to run. Much like counters has its own stage, we've move MV
mutations to their own stage, as well as batchlog mutations. We also made the tasks reschedule
themselves in the stage if they are waiting for a lock and cannot acquire it, so that the
stage doesn't deadlock.

These changes have improved MV so that we can overwrite a single key many times and get a
consistent result back at the end of the run. We have a [dtest|https://github.com/carlyeks/cassandra-dtest/tree/materialized-views]
which tests that scenario.

Collections included in the view are properly handled now as well, so the range tombstones
are created for the view properly; all of the types have now been verified by a unit test.

The next steps are to rebase this on top of 8099 for 3.0.

Many thanks to [~tjake] for all of his help looking into these problems.


> Materialized Views (was: Global Indexes)
> ----------------------------------------
>
>                 Key: CASSANDRA-6477
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6477
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: API, Core
>            Reporter: Jonathan Ellis
>            Assignee: Carl Yeksigian
>              Labels: cql
>             Fix For: 3.0 beta 1
>
>
> Local indexes are suitable for low-cardinality data, where spreading the index across
the cluster is a Good Thing.  However, for high-cardinality data, local indexes require querying
most nodes in the cluster even if only a handful of rows is returned.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message