cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Dinesh Joshi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-14647) Reading cardinality from Statistics.db failed
Date Thu, 16 Aug 2018 22:18:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-14647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583121#comment-16583121
] 

Dinesh Joshi commented on CASSANDRA-14647:
------------------------------------------

Hey Vitali, thanks for the report. [~krummas] any idea what is going on here? Would changing
compaction strategies cause this issue?

> Reading cardinality from Statistics.db failed
> ---------------------------------------------
>
>                 Key: CASSANDRA-14647
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-14647
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Compaction
>         Environment: Clients are doing only writes with Local One, cluster consist of
3 regions with RF3.
> Storage is configured wth jbod/XFS on 10 x 1Tb disks
> IOPS limit for each disk 500 (total 5000 iops)
> Bandwith for each disk 60mb/s (600 total)
> OS is Debian linux.
>            Reporter: Vitali Djatsuk
>            Priority: Major
>             Fix For: 3.0.x
>
>         Attachments: cassandra_compaction_pending_tasks_7days.png
>
>
> There is some issue with sstable metadata which is visible in system.log, the messages
says:
> {noformat}
> WARN  [Thread-6] 2018-07-25 07:12:47,928 SSTableReader.java:249 - Reading cardinality
from Statistics.db failed for /opt/data/disk5/data/keyspace/table/mc-big-Data.db.{noformat}
> Although there is no such file. 
> The message has appeared after i've changed the compaction strategy from SizeTiered to
Leveled. Compaction strategy has been changed region by region (total 3 regions) and it has
coincided with the double client write traffic increase.
>  I have tried to run nodetool scrub to rebuilt the sstable, but that does not fix the
issue.
> So very hard to define the steps to reproduce, probably it will be:
>  # run stress tool with write traffic
>  # under load change compaction strategy from SireTiered to Leveled for the bunch of
hosts
>  # add more write traffic
> Reading the code it is said that if this metadata is broken, then "estimating the keys
will be done using index summary". 
>  [https://github.com/apache/cassandra/blob/cassandra-3.0.17/src/java/org/apache/cassandra/io/sstable/format/SSTableReader.java#L247]
>   



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message