cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefania (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-10421) Potential issue with LogTransaction as it only checks in a single directory for files
Date Fri, 09 Oct 2015 04:14:26 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14949867#comment-14949867
] 

Stefania commented on CASSANDRA-10421:
--------------------------------------

Just outlining some more options:

* The current approach is the most permissive approach, in that we clean up a transaction
even if some final files are missing (because a disk was removed for example). My reasoning
is that from the transaction point of view, it did complete or abort and so it should not
keep old files if the new files have been removed for another reason, or should it?

* If we instead wrote all sstable files to all log files we could detect if some final files
are missing and refuse to clean-up. This would be safer but it does however have the disadvantage
that we could delete files in a folder even if in that folder there is no txn log. Currently
this is incompatible with listing temporary files as we only look at the txn file in the current
folder. So this could be counter-intuitive in that an operator might delete the txn log file
in a folder, confirm with {{sstableutil}} that the stables in that folder are not temporary,
and yet we delete those files on startup. So if we went down this route we'd have to extend
listing as well, at least when offline.

* There is one more approach, the most pessimistic, which is to refuse to clean up a transaction
if any log files are  missing. This we could detect by counting the number of records or log
files and adding this number to the final record, protected by the checksum. It would however
mean that if a user removed a disk, we would not clean-up the leftovers on a different disk
but we could stash them or let the operation handle them manually by printing out all necessary
information (CASSANDRA-10112).

> Potential issue with LogTransaction as it only checks in a single directory for files
> -------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-10421
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10421
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Marcus Eriksson
>            Assignee: Stefania
>            Priority: Blocker
>             Fix For: 3.0.0 rc2
>
>
> When creating a new LogTransaction we try to create the new logfile in the same directory
as the one we are writing to, but as we use {{[directories.getDirectoryForNewSSTables()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java#L125]}}
this might end up in "any" of the configured data directories. If it does, we will not be
able to clean up leftovers as we check for files in the same directory as the logfile was
created: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogRecord.java#L163
> cc [~Stefania]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message