cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stefania (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-10421) Potential issue with LogTransaction as it only checks in a single directory for files
Date Fri, 16 Oct 2015 01:08:05 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-10421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14959979#comment-14959979
] 

Stefania commented on CASSANDRA-10421:
--------------------------------------

bq. So what I think I see is that when the LogTransaction completes it first writes to the
log the commit record, and then starts making permanent changes to the the files on disk (deleting
the old ones). But if it hasn't actually synced the log to disk then on a restart we could
have a partial log and attempt to roll back, but it is too late because before the crash we
had already deleted parts of the before state. At the end we should sync the log files before
deleting the obsolete files right?

Yes this is correct, that's why we flush after writing every record but if we want to survive
a power cut then we should call {{fsync}}. 

bq. Was the intent to sync the folder when creating the log file or when adding a record which
indicates the addition of other data files?

The intent was to sync the folder when creating the file and to sync the content of the file
with a flush when appending data to it. However flushing only passes the data to the operating
system; it won't protect us from a power cut. This wasn't clear to me yesterday. At a minimum
we should {{fsync}} after adding the final record and probably also when adding new records
as you correctly pointed out. Therefore, I would strongly prefer to leave it as it was before
and call {{fsync}} on the folder whenever we add a record, so in case of a power cut we have
the most up-to-date data. This is what I did in the latest commit. Let me know if you have
performance concerns.

I also had to add several {{log.txnFile().close();}} to the unit tests (whenever we test {{removeUnfinishedLeftovers}})
because on Windows we cannot delete files that are open. This is a bit ugly so maybe we should
also go back to using {{FileUtils::appendLine}}, unless again you have performance concerns.

I've rebased on 3.0 and started a new CI run on both Linux and Windows.

> Potential issue with LogTransaction as it only checks in a single directory for files
> -------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-10421
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-10421
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Marcus Eriksson
>            Assignee: Stefania
>            Priority: Blocker
>             Fix For: 3.0.0 rc2
>
>
> When creating a new LogTransaction we try to create the new logfile in the same directory
as the one we are writing to, but as we use {{[directories.getDirectoryForNewSSTables()|https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogTransaction.java#L125]}}
this might end up in "any" of the configured data directories. If it does, we will not be
able to clean up leftovers as we check for files in the same directory as the logfile was
created: https://github.com/apache/cassandra/blob/trunk/src/java/org/apache/cassandra/db/lifecycle/LogRecord.java#L163
> cc [~Stefania]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message