cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joshua McKenzie (JIRA)" <>
Subject [jira] [Commented] (CASSANDRA-8743) NFS doesn't behave on Windows
Date Thu, 27 Aug 2015 22:13:49 GMT


Joshua McKenzie commented on CASSANDRA-8743:

So after doing a bit more thinking on this and looking at the code, we're failing not within
the C* code but rather on the attempt to memory-map the segment:
Caused by: Invalid argument
	at Method) ~[na:1.8.0_45]
	at ~[na:1.8.0_45]
	at org.apache.cassandra.db.commitlog.CommitLogSegment.<init>(

This got me to thinking - if the problem is memory mapping From the guest os To the host os,
that makes me suspicious of whatever mechanism virtualbox is using to mount/share those file
systems as I've tested memory-mapped i/o over NFS without issue. Digging around in virtualbox's
bug list a bit turns [this|] up.

TL;DR: memory-mapped files over vboxvfs (i.e. guest attempting to memory-map files on the
host filesystem) has been a problem for the past 8 years or so without a merged fix yet. So
having a Docker image that stores data on the host OS over vboxvfs to a Windows host is going
to give you trouble w/Cassandra as we memory-map portions of our CommitLogs. Unfortunately,
that's a bug on their end.

> NFS doesn't behave on Windows
> -----------------------------
>                 Key: CASSANDRA-8743
>                 URL:
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>            Reporter: Tamar Nirenberg
>            Assignee: Joshua McKenzie
>            Priority: Minor
>             Fix For: 2.2.x
>         Attachments: docker-system.log
> Running repair over NFS in Cassandra 2.1.2 encounters this error and crashes the ring:
> ERROR [ValidationExecutor:2] 2015-01-22 11:48:14,811 - Failed creating
a merkle tree for [repair #c84c7c70-a21b-11e4-aeca-19e6d7fa2595 on ATTRIBUTES/LINKS, (11621838520493020277529637175352775759,11853478749048239324667887059881170862]],
/ (see log for details)
> ERROR [ValidationExecutor:2] 2015-01-22 11:48:14,827 - Exception
in thread Thread[ValidationExecutor:2,1,main]
> java.nio.file.DirectoryNotEmptyException: /exlibris/cassandra/local/data/data/ATTRIBUTES/LINKS/snapshots/c84c7c70-a21b-11e4-aeca-19e6d7fa2595
>         at
>         at
>         at org.apache.cassandra.db.Directories.clearSnapshot( ~[apache-cassandra-2.1.2.jar:2.1.2]
>         at org.apache.cassandra.db.ColumnFamilyStore.clearSnapshot(
>         at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(
>         at org.apache.cassandra.db.compaction.CompactionManager.access$600(
>         at org.apache.cassandra.db.compaction.CompactionManager$
>         at ~[na:1.7.0_71]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(
>         at java.util.concurrent.ThreadPoolExecutor$
>         at [na:1.7.0_71]
> Caused by: java.nio.file.DirectoryNotEmptyException: /exlibris/cassandra/local/data/data/ATTRIBUTES/LINKS/snapshots/c84c7c70-a21b-11e4-aeca-19e6d7fa2595
>         at sun.nio.fs.UnixFileSystemProvider.implDelete(
>         at sun.nio.fs.AbstractFileSystemProvider.delete(
>         at java.nio.file.Files.delete( ~[na:1.7.0_71]
>         at
>         ... 10 common frames omitted
> ERROR [ValidationExecutor:2] 2015-01-22 11:48:14,829 - Stopping
> WARN  [ValidationExecutor:2] 2015-01-22 11:48:14,829 - Stopping
gossip by operator request
> INFO  [ValidationExecutor:2] 2015-01-22 11:48:14,829 - Announcing

This message was sent by Atlassian JIRA

View raw message