cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Blake Eggleston (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-13392) Repaired status should be cleared on new sstables when issuing nodetool refresh
Date Thu, 30 Mar 2017 23:14:41 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-13392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950041#comment-15950041
] 

Blake Eggleston edited comment on CASSANDRA-13392 at 3/30/17 11:13 PM:
-----------------------------------------------------------------------

This seems like a documentation problem. I don't think we should change the behavior of nodetool
refresh, it should be treated as a shortcut for stopping a node, adding sstables, and restarting
it. That's it. If an operator wants to add sstables to a live node we should expect that they’re
adding them in the correct state, and the nodetool docs should explicitly warn them that they
should modify repaired statuses prior to adding the sstables to the data directories. 

My reasoning here is:
* You each bring up valid use cases for nodetool refresh working one way or the other, so
we can't really make assumptions about the operator's intentions.
* If a node unexpectedly dies after the sstables have been added to the data directories,
it will load the sstables without adjusting any metatdata when it restarts, so the failure
recovery behavior would differ from the normal behavior pretty significantly.
* After (briefly) reviewing the CFS.loadNewSSTables code, along with the other addSSTable
code, I’m not confident that removing the repaired status of manually added sstables wouldn’t
also clear the repaired status of legit sstables inadvertently in some cases. Specifically,
there doesn’t appear to be anything preventing streamed repaired tables from appearing on
disk between when we get the initial set of sstables, and when we start scanning the files
on disk for sstables to add.





was (Author: bdeggleston):
This seems like a documentation problem. I don't think we should change the behavior of nodetool
refresh, it should be treated as a shortcut for stopping a node, adding sstables, and restarting
it. That's it. If an operator wants to add sstables to a live node we should expect that they’re
adding them in the correct state, and the nodetool docs should explicitly warn them that they
should modify repaired statuses prior to adding the sstables to the data set. 

My reasoning here is:
* You each bring up valid use cases for nodetool refresh working one way or the other, so
we can't really make assumptions about the operator's intentions.
* If a node unexpectedly dies after the sstables have been added to the dataset, it will load
the sstables without adjusting any metatdata when it restarts, so the failure recovery behavior
would differ from the normal behavior pretty significantly.
* After (briefly) reviewing the CFS.loadNewSSTables code, along with the other addSSTable
code, I’m not confident that removing the repaired status of manually added sstables wouldn’t
also clear the repaired status of legit sstables inadvertently in some cases. Specifically,
there doesn’t appear to be anything preventing streamed repaired tables from appearing on
disk between when we get the initial set of sstables, and when we start scanning the files
on disk for sstables to add.




> Repaired status should be cleared on new sstables when issuing nodetool refresh
> -------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-13392
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-13392
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Marcus Eriksson
>            Assignee: Marcus Eriksson
>             Fix For: 3.0.x, 3.11.x, 4.x
>
>
> We can't assume that new sstables added when doing nodetool refresh (ColumnFamilyStore#loadNewSSTables)
are actually repaired if they have the repairedAt flag set



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message