cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Jirsa (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-8460) Make it possible to move non-compacting sstables to slow/big storage in DTCS
Date Wed, 24 Jun 2015 02:02:43 GMT


Jeff Jirsa commented on CASSANDRA-8460:

Thanks for the feedback, [~krummas]!

Can't we check before starting an archive compaction if there are any archive locations available?
If there are none, we shouldn't compact, right?

Yea. There's a few cases here, and I suppose that answer works for all of them: 

- CF compaction strategy specifies archive tier, but no disk is configured on the node
- CF compaction strategy specifies archive tier, but there's no free space 
- If we were to allowe max_sstable_age_days > archive_sstables_age_days, there could be
a use case where 2 sstables on archive storage would be eligible for compaction, but there
may not be room for them to be combined. If we don't allow this, then the potential edge case
goes away.

I guess it could be a problem if users increase max_sstable_age_days and we move the data
back to the fast disks though, thoughts?

Is that a problem? If the user wants to tune the parameter, we should support it. 

As in 2), I think we should never compact the sstables on the slow disks.

I'll write it however you want it, but my assumption was that the {{max_sstable_age_days}}
parameter is set and greater than {{archive_sstables_age_days}}, we would still compact, it's
just obviously slower. In my mind, it's a cost/performance tradeoff for operators - slow disk
may not be SUPER slow, it may just be 10k iops instead of 20k iops, so compaction may be OK,
just not the best for hottest data. If you're adamant about not allowing compaction on the
archive tier, I'll add a check so that {{max_sstable_age_days}} can not be set higher than
{{archive_sstables_age_days}} . 

you should probably check absolute paths and use startsWith?

Noted, I like that way better.

Thanks again. I'll work on finishing this up and adding some tests.

> Make it possible to move non-compacting sstables to slow/big storage in DTCS
> ----------------------------------------------------------------------------
>                 Key: CASSANDRA-8460
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Marcus Eriksson
>            Assignee: Jeff Jirsa
>              Labels: dtcs
>             Fix For: 3.x
> It would be nice if we could configure DTCS to have a set of extra data directories where
we move the sstables once they are older than max_sstable_age_days. 
> This would enable users to have a quick, small SSD for hot, new data, and big spinning
disks for data that is rarely read and never compacted.

This message was sent by Atlassian JIRA

View raw message