cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Hatfield <nick.hatfi...@metricly.com>
Subject RE: Cassandra taking very long to start and server under heavy load
Date Thu, 02 May 2019 10:21:20 GMT
Just curious but, did you make sure to run the sstable upgrade after you completed the move
from 2.x to 3.x ?

From: Evgeny Inberg [mailto:evginb@gmail.com]
Sent: Thursday, May 02, 2019 1:31 AM
To: user@cassandra.apache.org
Subject: Re: Cassandra taking very long to start and server under heavy load

Using a sigle data disk.
Also, it is performing mostly heavy read operations according to the metrics cillected.
On Wed, 1 May 2019, 20:14 Jeff Jirsa <jjirsa@gmail.com<mailto:jjirsa@gmail.com>>
wrote:
Do you have multiple data disks?
Cassandra 6696 changed behavior with multiple data disks to make it safer in the situation
that one disk fails . It may be copying data to the right places on startup, can you see if
sstables are being moved on disk?
--
Jeff Jirsa


On May 1, 2019, at 6:04 AM, Evgeny Inberg <evginb@gmail.com<mailto:evginb@gmail.com>>
wrote:
I have upgraded a Cassandra cluster from version 2.0.x to 3.11.4 going trough 2.1.14.
After the upgrade, noticed that each node is taking about 10-15 minutes to start, and server
is under a very heavy load.
Did some digging around and got view leads from the debug log.
Messages like:
Keyspace.java:351 - New replication settings for keyspace system_auth - invalidating disk
boundary caches
CompactionStrategyManager.java:380 - Recreating compaction strategy - disk boundaries are
out of date for system_auth.roles.

This is repeating for all keyspaces.

Any suggestion to check and what might cause this to happen on every start?

Thanks!e
Mime
View raw message