cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Jirsa (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS
Date Fri, 12 Jun 2015 18:08:04 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14583809#comment-14583809
] 

Jeff Jirsa edited comment on CASSANDRA-8460 at 6/12/15 6:07 PM:
----------------------------------------------------------------

{quote}yes, I've been thinking maybe adding priorities or tags to the data directories, but
that is probably not needed now. Adding a flag to each data_directory that states whether
it is for archival storage or not is probably enough for now.{quote}

Asking for clarification to make sure I don't go too far into pony land:

So my initial approach was to define a second config item, separate from {{data_file_directories}}
entirely, so that no other code needed to be aware of it except for classes explicitly wanting
to use `archive` tier storage ( {{dd.getAllDataFileLocations()}} would not return the archive
tier, but rather add a {{dd.getArchiveDataFileLocations()}} specifically for the slow class
of storage).  

It sounds from your description you're envisioning changing the list of data_file_locations
to a map {noformat} tag1:location1,tag1:location2,tag3:location3 {noformat} or {noformat}
tag1:[location1,location2],tag3:[location3] {noformat} In this case, we'd also need to maintain
backwards compatibility, which seems fairly straight forward to do (check to see if the provided
{{data_files_directory}} is an old-format list rather than map and apply some default tag?)

The first approach is clean and isolated, unlikely to introduce surprises, but potentially
limits us from being able to do more interesting work with tagged data file directories later
(ie: only store data for KS W in data directories tagged X, and KS Y in data directories tagged
Z). Can you clarify which best fits your expectations? 



was (Author: jjirsa):
{quote}yes, I've been thinking maybe adding priorities or tags to the data directories, but
that is probably not needed now. Adding a flag to each data_directory that states whether
it is for archival storage or not is probably enough for now.{quote}

Asking for clarification to make sure I don't go too far into pony land:

So my initial approach was to define a second config item, separate from {{data_file_directories}}
entirely, so that no other code needed to be aware of it except for classes explicitly wanting
to use `archive` tier storage ( {{dd.getAllDataFileLocations()}} would not return the archive
tier, but rather add a {{dd.getArchiveDataFileLocations()}} specifically for the slow class
of storage).  

It sounds from your description you're envisioning changing the list of data_file_locations
to a map {noformat} [tag1:location1,tag1:location2,tag3:location3] {noformat} or {noformat}
tag1:[location1,location2],tag3:[location3] {noformat} In this case, we'd also need to maintain
backwards compatibility, which seems fairly straight forward to do (check to see if the provided
{{data_files_directory}} is an old-format list rather than map and apply some default tag?)

The first approach is clean and isolated, unlikely to introduce surprises, but potentially
limits us from being able to do more interesting work with tagged data file directories later
(ie: only store data for KS W in data directories tagged X, and KS Y in data directories tagged
Z). Can you clarify which best fits your expectations? 


> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> ----------------------------------------------------------------------------
>
>                 Key: CASSANDRA-8460
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8460
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Marcus Eriksson
>              Labels: dtcs
>
> It would be nice if we could configure DTCS to have a set of extra data directories where
we move the sstables once they are older than max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big spinning
disks for data that is rarely read and never compacted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message