cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-10421) Potential issue with LogTransaction as it only checks in a single directory for files
Date Mon, 12 Oct 2015 07:34:05 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14952746#comment-14952746
] 

Benedict edited comment on CASSANDRA-10421 at 10/12/15 7:33 AM:
----------------------------------------------------------------

FTR, IMO (as I expressed on IRC), I think it is both safer and easier to implement duplication
to all disks. Since this data is low traffic it also shouldn't be a major issue that we do
this:

Implementation-wise, the changes can be extremely minimal to the existing logic: on read,
we read all files simultaneously, line-by-line, corroborating they are the same (we can even
have a special reader that wraps all files, so that the current logic is completely unchanged).
If at any point they differ, we confirm there is no proceeding line (if there is we leave
everything as is, as it is corruption, much like we normally would), and otherwise we confirm
that every file's last line is a prefix-match of every other. On write we can do similar,
ensuring we write to all of them in sequence, treating it as a single action from the POV
of the existing logic. This means the only modifications are at the very outermost parts of
the logic, so we don't have to revisit the safety of the basic functionality.

This also leaves us with much less complexity reasoning about the safety under different scenarios
wrt disk corruption, failure, accidental user-error, etc. So, it seems like a win-win AFAICT.
The only downside is that users modifying these files would need to modify them all, but this
is not overly onerous.

(I will note I'm not convinced it _cannot_ be done safely with separate disks, just that it's
going to be fiddlier, and I haven't had time to consider it beyond that)


was (Author: benedict):
FTR, IMO (as I expressed on IRC), I think it is both safer and easier to implement duplication
to all disks. Since this data is low traffic it also shouldn't be a major issue that we do
this:

Implementation-wise, the changes can be extremely minimal to the existing logic: on read,
we read all files simultaneously, line-by-line, corroborating they are the same (we can even
have a special reader that wraps all files, so that the current logic is completely unchanged).
If at any point they differ, we confirm there is no proceeding line (if there is we leave
everything as is, as it is corruption, much like we normally would), and otherwise we confirm
that every file's last line is a prefix-match of every other. On write we can do similar,
ensuring we write to all of them in sequence, treating it as a single action from the POV
of the existing logic. This means the only modifications are at the very outermost parts of
the logic, so we don't have to revisit the safety of the basic functionality.

This also leaves us with much less complexity reasoning about the safety under different scenarios
wrt disk corruption, failure, accidental user-error, etc. So, it seems like a win-win AFAICT.
The only downside is that users modifying these files would need to modify them all, but this
is not overly onerous.


> Potential issue with LogTransaction as it only checks in a single directory for files
> -------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-10421
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10421
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Marcus Eriksson
>            Assignee: Stefania
>            Priority: Blocker
>             Fix For: 3.0.0 rc2
>
>
> When creating a new LogTransaction we try to create the new logfile in the same directory
as the one we are writing to, but as we use {{[directories.getDirectoryForNewSSTables()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java#L125]}}
this might end up in "any" of the configured data directories. If it does, we will not be
able to clean up leftovers as we check for files in the same directory as the logfile was
created: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogRecord.java#L163
> cc [~Stefania]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message