cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kurt Greaves (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-11163) Summaries are needlessly rebuilt when the BF FP ratio is changed
Date Fri, 02 Mar 2018 02:05:00 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-11163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383009#comment-16383009
] 

Kurt Greaves commented on CASSANDRA-11163:
------------------------------------------

Thanks for the review Chris. Made changes according to your comments and updated the branches.
# That works. It also didn't break tests because there was that initial catchall isOfflline
at the top which covered all those cases. Note that ATM the only way the summary is recreated
is by online operations (using open, rather than openNoValidation).
# No it wasn't, assumed in Java we only had access to second precision which is not true in
Java 8, as long as you use {{Files}}. I've updated the test to use 10ms sleeps with ms precision
and it seems to be much more reliable. 99% sure the rare failures before were just because
the second precision of {{lastModified()}}
# I've updated to just use {{isOffline}}. It seemed to make more sense than {{allowChanges}}
to me, in that "we don't track hotness or touch files if we're offline". I'll note that this
changed the behaviour of {{org.apache.cassandra.db.ColumnFamilyStore#getSnapshotSSTableReader}}
to be "offline", however I think that makes sense in this case, as we shouldn't be regenerating
summaries/BF's for snapshots anyway.

|[3.0|https://github.com/apache/cassandra/compare/cassandra-3.0...kgreav:14166-3.0]|[3.11|https://github.com/apache/cassandra/compare/cassandra-3.11...kgreav:14166-3.11]|[trunk|https://github.com/apache/cassandra/compare/trunk...kgreav:14166-trunk]|


> Summaries are needlessly rebuilt when the BF FP ratio is changed
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-11163
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-11163
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Brandon Williams
>            Assignee: Kurt Greaves
>            Priority: Major
>             Fix For: 3.0.x, 3.11.x, 4.x
>
>
> This is from trunk, but I also saw this happen on 2.0:
> Before:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 221460
> drwxr-xr-x 2 root root      4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root        80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root     26518 Feb 11 23:50 ma-6-big-Summary.db
> -rw-r--r-- 1 root root     10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root    192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root        10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root      2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root        80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root     26518 Feb 11 23:50 ma-7-big-Summary.db
> -rw-r--r-- 1 root root     10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root    192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root         9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root      2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root        80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root    104178 Feb 11 23:50 ma-5-big-Summary.db
> -rw-r--r-- 1 root root     10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root    757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root         9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root      8508 Feb 11 23:50 ma-5-big-CRC.db
> root@bw-1:/srv/cassandra# md5sum /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db
> 5fca154fc790f7cfa37e8ad6d1c7552c
> {noformat}
> BF ratio changed, node restarted:
> {noformat}
> root@bw-1:/srv/cassandra# ls -ltr /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/
> total 242168
> drwxr-xr-x 2 root root      4096 Feb 11 23:34 backups
> -rw-r--r-- 1 root root        80 Feb 11 23:50 ma-6-big-TOC.txt
> -rw-r--r-- 1 root root     10264 Feb 11 23:50 ma-6-big-Statistics.db
> -rw-r--r-- 1 root root   2607705 Feb 11 23:50 ma-6-big-Index.db
> -rw-r--r-- 1 root root    192440 Feb 11 23:50 ma-6-big-Filter.db
> -rw-r--r-- 1 root root        10 Feb 11 23:50 ma-6-big-Digest.crc32
> -rw-r--r-- 1 root root  35212125 Feb 11 23:50 ma-6-big-Data.db
> -rw-r--r-- 1 root root      2156 Feb 11 23:50 ma-6-big-CRC.db
> -rw-r--r-- 1 root root        80 Feb 11 23:50 ma-7-big-TOC.txt
> -rw-r--r-- 1 root root     10264 Feb 11 23:50 ma-7-big-Statistics.db
> -rw-r--r-- 1 root root   2607614 Feb 11 23:50 ma-7-big-Index.db
> -rw-r--r-- 1 root root    192432 Feb 11 23:50 ma-7-big-Filter.db
> -rw-r--r-- 1 root root         9 Feb 11 23:50 ma-7-big-Digest.crc32
> -rw-r--r-- 1 root root  35190400 Feb 11 23:50 ma-7-big-Data.db
> -rw-r--r-- 1 root root      2152 Feb 11 23:50 ma-7-big-CRC.db
> -rw-r--r-- 1 root root        80 Feb 11 23:50 ma-5-big-TOC.txt
> -rw-r--r-- 1 root root     10264 Feb 11 23:50 ma-5-big-Statistics.db
> -rw-r--r-- 1 root root  10289077 Feb 11 23:50 ma-5-big-Index.db
> -rw-r--r-- 1 root root    757384 Feb 11 23:50 ma-5-big-Filter.db
> -rw-r--r-- 1 root root         9 Feb 11 23:50 ma-5-big-Digest.crc32
> -rw-r--r-- 1 root root 139201355 Feb 11 23:50 ma-5-big-Data.db
> -rw-r--r-- 1 root root      8508 Feb 11 23:50 ma-5-big-CRC.db
> -rw-r--r-- 1 root root        80 Feb 12 00:03 ma-8-big-TOC.txt
> -rw-r--r-- 1 root root     14902 Feb 12 00:03 ma-8-big-Summary.db
> -rw-r--r-- 1 root root     10264 Feb 12 00:03 ma-8-big-Statistics.db
> -rw-r--r-- 1 root root   1458631 Feb 12 00:03 ma-8-big-Index.db
> -rw-r--r-- 1 root root     10808 Feb 12 00:03 ma-8-big-Filter.db
> -rw-r--r-- 1 root root        10 Feb 12 00:03 ma-8-big-Digest.crc32
> -rw-r--r-- 1 root root  19660275 Feb 12 00:03 ma-8-big-Data.db
> -rw-r--r-- 1 root root      1204 Feb 12 00:03 ma-8-big-CRC.db
> -rw-r--r-- 1 root root     26518 Feb 12 00:04 ma-7-big-Summary.db
> -rw-r--r-- 1 root root     26518 Feb 12 00:04 ma-6-big-Summary.db
> -rw-r--r-- 1 root root    104178 Feb 12 00:04 ma-5-big-Summary.db
> root@bw-1:/srv/cassandra# md5sum /var/lib/cassandra/data/keyspace1/standard1-071efdc0d11811e590c3413ee28a6c90/ma-5-big-Summary.db

> 5fca154fc790f7cfa37e8ad6d1c7552c 
> {noformat}
> This hurts startup time and appears to do nothing useful whatsoever.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscribe@cassandra.apache.org
For additional commands, e-mail: commits-help@cassandra.apache.org


Mime
View raw message