cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Melvin Kanasseril <Melvin.Kanasse...@Sophos.com>
Subject Restoring from snapshot after index creation fails
Date Tue, 25 Jul 2017 15:12:33 GMT
Hi folks,

We have a cluster with Cassandra 3.9, running on AWS in a single region and multi AZ setup.
We use Priam do our backup to S3 and auto restore on node bootstrap. We created an index on
a simple uuid field on one of our existing tables, after which the restores fail on that table
(see log messages below) whenever a node is cycled.

This causes the Cassandra service to exit hard and the node fails to bootstrap. The restore
used to worked fine before this and gives no errors on the other tables. Has anybody seen
this issue before? Is this a bug with snapshotting?

To get the node up and running we removed the table files and executed a full repair after.
Our RF is 3. Do we also have to do a nodetool rebuild_index?

Thanks,
Melvin

Jul 22 00:07:58 ip-123-12-123-123 su[22260]: INFO  00:07:58 Initializing path.redacted_table_name
Jul 22 00:07:58 ip-123-12-123-123 su[22260]: ERROR 00:07:58 Cannot open /path/path/redacted_table_name-88308780560011e7a7e7dba00a2b2866/mc-181-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system partitioner org.apache.cassandra.dht.Murmur3Partitioner.
 Note that the default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
Jul 22 00:07:58 ip-123-12-123-123 su[22260]: ERROR 00:07:58 Cannot open /path/path/redacted_table_name-88308780560011e7a7e7dba00a2b2866/mc-204-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system partitioner org.apache.cassandra.dht.Murmur3Partitioner.
 Note that the default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
Jul 22 00:07:58 ip-123-12-123-123 su[22260]: ERROR 00:07:58 Cannot open /path/path/redacted_table_name-88308780560011e7a7e7dba00a2b2866/mc-185-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system partitioner org.apache.cassandra.dht.Murmur3Partitioner.
 Note that the default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
Jul 22 00:07:58 ip-123-12-123-123 su[22260]: ERROR 00:07:58 Cannot open /path/path/redacted_table_name-88308780560011e7a7e7dba00a2b2866/mc-177-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system partitioner org.apache.cassandra.dht.Murmur3Partitioner.
 Note that the default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
Jul 22 00:07:58 ip-123-12-123-123 su[22260]: ERROR 00:07:58 Cannot open /path/path/redacted_table_name-88308780560011e7a7e7dba00a2b2866/mc-101-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system partitioner org.apache.cassandra.dht.Murmur3Partitioner.
 Note that the default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
Jul 22 00:07:58 ip-123-12-123-123 su[22260]: ERROR 00:07:58 Cannot open /path/path/redacted_table_name-88308780560011e7a7e7dba00a2b2866/mc-192-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system partitioner org.apache.cassandra.dht.Murmur3Partitioner.
 Note that the default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
Jul 22 00:07:58 ip-123-12-123-123 su[22260]: ERROR 00:07:58 Cannot open /path/path/redacted_table_name-88308780560011e7a7e7dba00a2b2866/mc-212-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system partitioner org.apache.cassandra.dht.Murmur3Partitioner.
 Note that the default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
Jul 22 00:07:58 ip-123-12-123-123 su[22260]: ERROR 00:07:58 Cannot open /path/path/redacted_table_name-88308780560011e7a7e7dba00a2b2866/mc-216-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system partitioner org.apache.cassandra.dht.Murmur3Partitioner.
 Note that the default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
Jul 22 00:07:58 ip-123-12-123-123 su[22260]: ERROR 00:07:58 Cannot open /path/path/redacted_table_name-88308780560011e7a7e7dba00a2b2866/mc-208-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system partitioner org.apache.cassandra.dht.Murmur3Partitioner.
 Note that the default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
Jul 22 00:07:58 ip-123-12-123-123 su[22260]: ERROR 00:07:58 Cannot open /path/path/redacted_table_name-88308780560011e7a7e7dba00a2b2866/mc-188-big;
partitioner org.apache.cassandra.dht.LocalPartitioner does not match system partitioner org.apache.cassandra.dht.Murmur3Partitioner.
 Note that the default partitioner starting with Cassandra 1.2 is Murmur3Partitioner, so you
will need to edit that to match your old partitioner if upgrading.
Jul 22 00:07:59 ip-123-12-123-123 systemd[1]: cassandra.service: Main process exited, code=exited,
status=1/FAILURE
Jul 22 00:07:59 ip-123-12-123-123 systemd[1]: cassandra.service: Unit entered failed state.
Jul 22 00:07:59 ip-123-12-123-123 systemd[1]: cassandra.service: Failed with result 'exit-code'.
Jul 22 00:07:59 ip-123-12-123-123 systemd[1]: Stopping User Manager for UID 997...
Mime
View raw message