cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tyler Hobbs (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5515) Track sstable coldness
Date Fri, 19 Jul 2013 16:04:49 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13713789#comment-13713789
] 

Tyler Hobbs commented on CASSANDRA-5515:
----------------------------------------

bq. I think we do need to persist this one across restarts, e.g., in a system table. (Maybe
just a Map would be fine, since pulling 1000s of entries into memory once on startup to initialize
state should be okay.)

I take it we will want to periodically sync the system table with the latest counts?  (Just
relying on some kind of shutdown hook seems like a bad idea.)  Given that there can be 10's
of thousands of sstables, I think we'll want to throttle that sync.

By "a Map", I assume you mean a map column collection.  I am a little concerned about pulling
in a map with 10's of thousands of entries all at once, even if it's just on startup.

As far clearing entries goes, directly deleting when the sstable is removed combined with
a TTL of 10 days to handle odd cases seems reasonable to me.

Thoughts?
                
> Track sstable coldness
> ----------------------
>
>                 Key: CASSANDRA-5515
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5515
>             Project: Cassandra
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Jonathan Ellis
>            Assignee: Tyler Hobbs
>             Fix For: 2.0.1
>
>         Attachments: 0001-Track-row-read-counts-in-SSTR.patch
>
>
> Keeping a count of reads per-sstable would allow STCS to automatically ignore cold data
rather than recompacting it constantly with hot data, dramatically reducing compaction load
for typical time series applications and others with time-correlated access patterns.  We
would not need a separate age-tiered compaction strategy.
> (This will really be useful in conjunction with CASSANDRA-5514.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message