cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CASSANDRA-2346) Windows: SSTableWriterTest Fails
Date Fri, 18 Mar 2011 18:34:29 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13008566#comment-13008566
] 

Jonathan Ellis commented on CASSANDRA-2346:
-------------------------------------------

My guess is, this part of closeAndOpenReader just prior to the delete attempt is responsible:

{code}
        SegmentedFile ifile = iwriter.builder.complete(newdesc.filenameFor(SSTable.COMPONENT_INDEX));
        SegmentedFile dfile = dbuilder.complete(newdesc.filenameFor(SSTable.COMPONENT_DATA));
        SSTableReader sstable = SSTableReader.internalOpen(newdesc, components, metadata,
partitioner, ifile, dfile, iwriter.summary, iwriter.bf, maxDataAge, estimatedRowSize, estimatedColumnCount);
{code}

In particular, iwriter.builder.complete mmap's the index file, which for Windows also counts
as having an open fd.  A reference to the mmap'd segments is retained by the returned sstable.

A nasty hack that MIGHT work (depending on GC to clean things up is awful) would be to change
the test to something like

{code}
        // avoid retaining a reference to the returned sstable
        Descriptor descriptor = SSTableUtils.prepare().cf("Indexed1").writeRaw(entries).descriptor;
       
        // clean up mmap'd segments in the reader we just created so Windows allows deleting
the backing file
        System.gc();
        // whack the index to trigger the recover
        FileUtils.deleteWithConfirm(descriptor.filenameFor(Component.PRIMARY_INDEX));
        FileUtils.deleteWithConfirm(descriptor.filenameFor(Component.FILTER));
{code}

A cleaner approach would be to ditch SSTableUtils (not my favorite class to begin with), expose
a method in SSTWriter that just does data appends (no index) and create just the data file
that way, instead of trying to delete the supporting files later.

An in-between approach might be to create a SSTW close() method that just returns the descriptor
after shutting things down, without opening the problematic reader.  Then the test would work
as-is.

> Windows: SSTableWriterTest Fails
> --------------------------------
>
>                 Key: CASSANDRA-2346
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2346
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Tests
>    Affects Versions: 0.7.4
>         Environment: Windows
>            Reporter: Benjamin Coverston
>              Labels: windows
>             Fix For: 0.7.5
>
>
> Fails with the following exception[1]
> Which looks like there's a stream keeping this file open somewhere.
> I looked for the culprit, but I was unable to identify it. Following it through it appears
that if the following two lines of code are executed from IndexWriter.afterAppend:
>             ByteBufferUtil.writeWithShortLength(key.key, indexFile);
>             indexFile.writeLong(dataPosition);
> Then the indexFile gets touched and that generally triggers this situation. One thing
interesting here is that an index file without any rows will trigger a divide by zero exception
[2].
> I looked at this quite a bit without being able to make much progress.
> [1]
> java.io.IOException: Failed to delete C:\Users\Ben\AppData\Local\Temp\Keyspace12505092819180437091Indexed1\Keyspace1\Indexed1-f-0-Index.db
> 	at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:51)
> 	at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:41)
> 	at org.apache.cassandra.io.sstable.SSTableWriterTest.testRecoverAndOpen(SSTableWriterTest.java:79)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
> 	at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
> 	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:97)
> 	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:196)
> 	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:115)
> [2]
> ERROR 16:57:48,119 Error in ThreadPoolExecutor
> java.lang.ArithmeticException: / by zero
> 	at org.apache.cassandra.io.sstable.SSTable.estimateRowsFromData(SSTable.java:218)
> 	at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:291)
> 	at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:942)
> 	at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:935)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:619)
> java.util.concurrent.ExecutionException: java.lang.ArithmeticException: / by zero
> 	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
> 	at java.util.concurrent.FutureTask.get(FutureTask.java:83)
> 	at org.apache.cassandra.io.sstable.SSTableWriterTest.testRecoverAndOpen(SSTableWriterTest.java:82)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:41)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:44)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:180)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:41)
> 	at org.junit.runners.ParentRunner$1.evaluate(ParentRunner.java:173)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:28)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:31)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:220)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:159)
> 	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:97)
> 	at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:196)
> 	at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:65)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> 	at com.intellij.rt.execution.application.AppMain.main(AppMain.java:115)
> Caused by: java.lang.ArithmeticException: / by zero
> 	at org.apache.cassandra.io.sstable.SSTable.estimateRowsFromData(SSTable.java:218)
> 	at org.apache.cassandra.io.sstable.SSTableWriter$Builder.build(SSTableWriter.java:291)
> 	at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:942)
> 	at org.apache.cassandra.db.CompactionManager$9.call(CompactionManager.java:935)
> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:619)

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message