cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Knighton (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-10621) Error while bootstraping a new node with Materialized Views
Date Fri, 30 Oct 2015 03:23:27 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14981810#comment-14981810
] 

Joel Knighton edited comment on CASSANDRA-10621 at 10/30/15 3:22 AM:
---------------------------------------------------------------------

Thanks, Alan.

This error is thrown when a node is bootstrapping and is streamed a mutation that affects
a MV base table and the token of the view partition key is not owned by the bootstrapping
node.

It was not caught in dtests because the base table and view table always had the same value
for the partition key in writes, so there was always a pending node for the view token.

I've pushed a fix as well as changes to the dtests to better detect similar issues. I've also
added a dtest for write survey mode. I'll update once CassCi has results.

----
* [dtest commit|https://github.com/jkni/cassandra-dtest/tree/74125f3f218a6eb7ff6403e8d21e91c48bfff2ab]
* [Cassandra 3.0 commit|https://github.com/jkni/cassandra/tree/b21a19b3131de49b01c615953f04b7092f03f638]
* [cassci-dtest|http://cassci.datastax.com/job/jkni-10621-dtest/]
* [cassci-testall|http://cassci.datastax.com/job/jkni-10621-testall/]


was (Author: jkni):
Thanks, Alan.

This error is thrown when a node is bootstrapping and is streamed a mutation that affects
a MV base table and the token of the view partition key is not owned by the bootstrapping
node.

It was not caught in dtests because the base table and view table always had the same value
for the partition key in writes, so there was always a pending node for the view token.

I've pushed a fix as well as changes to the dtests to better detect similar issues. I've also
added a dtest for write survey mode.

----
* [dtest commit|https://github.com/jkni/cassandra-dtest/tree/74125f3f218a6eb7ff6403e8d21e91c48bfff2ab]
* [Cassandra 3.0 commit|https://github.com/jkni/cassandra/tree/b21a19b3131de49b01c615953f04b7092f03f638]
* [cassci-dtest|http://cassci.datastax.com/job/jkni-10621-dtest/]
* [cassci-testall|http://cassci.datastax.com/job/jkni-10621-testall/]

> Error while bootstraping a new node with Materialized Views
> -----------------------------------------------------------
>
>                 Key: CASSANDRA-10621
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10621
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Alan Boudreault
>            Assignee: Joel Knighton
>            Priority: Critical
>             Fix For: 3.0.0
>
>         Attachments: system.log
>
>
> If I try to add a new node in a cluster that has materialized views, I get the following
error:
> {code}
>  ERROR 19:05:15 Unknown exception caught while attempting to update MaterializedView!
mview.user_playlists
> java.lang.RuntimeException: Trying to get the view natural endpoint on a non-data replica
>         at org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
~[main/:na]
>         at org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
~[main/:na]
>         at org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
~[main/:na]
>         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482) [main/:na]
>         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387) [main/:na]
>         at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215) [main/:na]
>         at org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
[main/:na]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 19:05:15 Error applying streamed sstable: 
> java.lang.RuntimeException: Trying to get the view natural endpoint on a non-data replica
>         at org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
~[main/:na]
>         at org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
~[main/:na]
>         at org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
~[main/:na]
>         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482) ~[main/:na]
>         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387) ~[main/:na]
>         at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215) ~[main/:na]
>         at org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
~[main/:na]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> WARN  19:05:18 Writing large partition mview/genre_to_user:genre_3 (116986142 bytes)
> WARN  19:05:21 Writing large partition mview/genre_to_user:genre_2 (116985746 bytes)
> WARN  19:05:24 Writing large partition mview/genre_to_user:genre_5 (116986337 bytes)
> ERROR 19:05:33 Unknown exception caught while attempting to update MaterializedView!
mview.user_playlists
> java.lang.RuntimeException: Trying to get the view natural endpoint on a non-data replica
>         at org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
~[main/:na]
>         at org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
~[main/:na]
>         at org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
~[main/:na]
>         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482) [main/:na]
>         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387) [main/:na]
>         at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215) [main/:na]
>         at org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
[main/:na]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 19:05:33 Error applying streamed sstable: 
> java.lang.RuntimeException: Trying to get the view natural endpoint on a non-data replica
>         at org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
~[main/:na]
>         at org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
~[main/:na]
>         at org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
~[main/:na]
>         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482) ~[main/:na]
>         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387) ~[main/:na]
>         at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215) ~[main/:na]
>         at org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
~[main/:na]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 19:05:40 Unknown exception caught while attempting to update MaterializedView!
mview.user_playlists
> java.lang.RuntimeException: Trying to get the view natural endpoint on a non-data replica
>         at org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
~[main/:na]
>         at org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
~[main/:na]
>         at org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
~[main/:na]
>         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482) [main/:na]
>         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387) [main/:na]
>         at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215) [main/:na]
>         at org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
[main/:na]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> ERROR 19:05:40 Error applying streamed sstable: 
> java.lang.RuntimeException: Trying to get the view natural endpoint on a non-data replica
>         at org.apache.cassandra.db.view.ViewUtils.getViewNaturalEndpoint(ViewUtils.java:103)
~[main/:na]
>         at org.apache.cassandra.service.StorageProxy.mutateMV(StorageProxy.java:693)
~[main/:na]
>         at org.apache.cassandra.db.view.ViewManager.pushViewReplicaUpdates(ViewManager.java:145)
~[main/:na]
>         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:482) ~[main/:na]
>         at org.apache.cassandra.db.Keyspace.apply(Keyspace.java:387) ~[main/:na]
>         at org.apache.cassandra.db.Mutation.applyUnsafe(Mutation.java:215) ~[main/:na]
>         at org.apache.cassandra.streaming.StreamReceiveTask$OnCompletionRunnable.run(StreamReceiveTask.java:162)
~[main/:na]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_45]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
[na:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
[na:1.8.0_45]
>         at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message