jackrabbit-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bernd Eckenfels (JIRA)" <j...@apache.org>
Subject [jira] Commented: (JCR-2819) FileDataStore not crash safe
Date Thu, 02 Dec 2010 23:54:11 GMT

    [ https://issues.apache.org/jira/browse/JCR-2819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12966353#action_12966353

Bernd Eckenfels commented on JCR-2819:

> As you may know, there was a discussion about how a file system is supposed to work

It was new to me, that it finally got some attention, but it sounds limited to ext4 - and
it sounds like it is only covering the "close/rename(replace)" not "close/rename(new)" case.
But anyway... NTFS or XFS are other systems to cover.

> In many systems (specially clustered environments) it doesn't make sense to ensure the
data is written to disk, because the system is anyway crash proof. 

I dont understand this. Do you mean a Jackrabit cluster? In that case as far as I know the
File Datastore has no special replication feature and will crash. On hardware clusters Crashes
are also possible.

> it's important that the repository is in a consistent state

The problem is, if the fsync was not happening you get empty files under a hash. Which means
that the repository is not consistent anymore and all new documents with the same hash give
you a collission alert. So it is a matter of consitency.

I agree that it might not be needed to make fsync the default, but users who care about transactional
safety should be able to find the switch.

> FileDataStore not crash safe
> ----------------------------
>                 Key: JCR-2819
>                 URL: https://issues.apache.org/jira/browse/JCR-2819
>             Project: Jackrabbit Content Repository
>          Issue Type: Bug
>          Components: jackrabbit-core
>    Affects Versions: 2.1.2
>         Environment: All
>            Reporter: Bernd Eckenfels
>            Priority: Minor
>         Attachments: FileDataStore.patch
> The FileDataStore.addRecord() does create a temporary file and rename it to the final
location. On a crash, this can result in a file which is renamed but the content is still
empty. This especially happens on systems where data in the filesystem is not journaled. You
can resolve this problem by calling the fdsync() method of the operating system before renaming
the file. Typically in Java this is done by first flushing all the Java buffers (i.e. calling
flush() on the highest output stream), and then using the getFD().sync(); method on the FileOutputStream.
> Note: this introduced some delay/additional IO load on the system, therefore I think
it might be best to make it configurable. But I think in some environments the additional
reliability is badly needed.
> Sample patch without configuration parameter added.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message