cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kurt Greaves (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-12783) Break up large MV mutations to prevent OOMs
Date Mon, 21 Aug 2017 12:04:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16135083#comment-16135083
] 

Kurt Greaves commented on CASSANDRA-12783:
------------------------------------------

[trunk|https://github.com/apache/cassandra/compare/trunk...kgreav:trunk-11670?expand=1]

BatchlogManager is more or less done which is the main component. Haven't fixed {{LegacyBatchlogMigrator}}
yet so plz ignore (the version there is from the last batchlog migration). Still got a bit
of work to do on tests, but will continue with that after sorting out UUID issue. For interests
sake, in the test {{testUUIDComparison}} id1 will occasionally return a UUID that is <
id2, despite id2 having a timestamp (and uuid for that matter) that is far lower. This is
a problem  [here|https://github.com/apache/cassandra/compare/trunk...kgreav:trunk-11670?expand=1#diff-c77d5f1027ee5e8a49e106903a4ca937R318]
the code will occasionally delete a batch that shouldn't be expired.

Also totally open to ideas/criticism regarding the whole design, specifically the expiry stuff
as not sure yet if that will have other implications.

cc [~pauloricardomg] if you want to have a look

> Break up large MV mutations to prevent OOMs
> -------------------------------------------
>
>                 Key: CASSANDRA-12783
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12783
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Local Write-Read Paths, Materialized Views
>            Reporter: Carl Yeksigian
>            Assignee: Kurt Greaves
>             Fix For: 4.x
>
>
> We only use the code path added in CASSANDRA-12268 for the view builder because otherwise
we would break the contract of the batchlog, where some mutations may be written and pushed
out before the whole batch log has been saved.
> We would need to ensure that all of the updates make it to the batchlog before allowing
the batchlog manager to try to replay them, but also before we start pushing out updates to
the paired replicas.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message