cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andreas Schnitzerling (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-9686) FSReadError and LEAK DETECTED after upgrading
Date Thu, 02 Jul 2015 09:06:04 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14611681#comment-14611681
] 

Andreas Schnitzerling commented on CASSANDRA-9686:
--------------------------------------------------

I checked the 2.1.7 instance, which I copied before upgrading. There is the same issue and
I have an idea: Our computers are in 3 different laboratories for electronic engineers. C*
is running there as a backround-job. We have regular emergency-tests once a month in the morning.
They switch off power and computers are not shut-down. I think I cannot change the process
there and in laboratories it can always happen during electronic-tests, that fuses are triggered.
That can happen anywhere since not everybody uses power-backup. For my opinion, invalid files
like here should be deleted - maybe with a warning in the log - especially if we don't loose
data like here for temporary process-infos.
{code:title=system.log v2.1.7}
ERROR [SSTableBatchOpen:1] 2015-06-24 14:12:02,033 CassandraDaemon.java:223 - Exception in
thread Thread[SSTableBatchOpen:1,5,main]
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file with 0 chunks encountered:
java.io.DataInputStream@1a32dcf
	at org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:205)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.compress.CompressionMetadata.<init>(CompressionMetadata.java:127)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:85)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:79)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:72)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:168) ~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:721) ~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:676) ~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:482) ~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:381) ~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:519) ~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.7.0_55]
	at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_55]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_55]
	at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: java.io.IOException: Compressed file with 0 chunks encountered: java.io.DataInputStream@1a32dcf
	at org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:183)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	... 15 common frames omitted
ERROR [SSTableBatchOpen:1] 2015-06-24 14:12:02,123 CassandraDaemon.java:223 - Exception in
thread Thread[SSTableBatchOpen:1,5,main]
org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file with 0 chunks encountered:
java.io.DataInputStream@11ca50a
	at org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:205)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.compress.CompressionMetadata.<init>(CompressionMetadata.java:127)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:85)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:79)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:72)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:168) ~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:721) ~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.sstable.SSTableReader.load(SSTableReader.java:676) ~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:482) ~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.sstable.SSTableReader.open(SSTableReader.java:381) ~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at org.apache.cassandra.io.sstable.SSTableReader$4.run(SSTableReader.java:519) ~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.7.0_55]
	at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_55]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_55]
	at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
Caused by: java.io.IOException: Compressed file with 0 chunks encountered: java.io.DataInputStream@11ca50a
	at org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:183)
~[apache-cassandra-2.1.7-SNAPSHOT.jar:2.1.7-SNAPSHOT]
	... 15 common frames omitted
ERROR [Reference-Reaper:1] 2015-06-24 14:12:02,593 Ref.java:179 - LEAK DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@111283) to class org.apache.cassandra.io.util.PoolingSegmentedFile$Cleanup@714273:C:\VirtualDrives\D\Programme\cassandra\data\system\sstable_activity\system-sstable_activity-ka-1597-Index.db
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-06-24 14:12:02,593 Ref.java:179 - LEAK DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@185571) to class org.apache.cassandra.io.sstable.SSTableReader$InstanceTidier@26308560:C:\VirtualDrives\D\Programme\cassandra\data\system\sstable_activity\system-sstable_activity-ka-1597
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-06-24 14:12:02,593 Ref.java:179 - LEAK DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@3eaef5) to class org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@22054172:[Memory@[0..4),
Memory@[0..18)] was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-06-24 14:12:02,593 Ref.java:179 - LEAK DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@936f7e) to class org.apache.cassandra.utils.concurrent.WrappedSharedCloseable$1@24953003:[Memory@[0..14),
Memory@[0..e1)] was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-06-24 14:12:02,593 Ref.java:179 - LEAK DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@1afb805) to class org.apache.cassandra.io.sstable.SSTableReader$InstanceTidier@22709880:C:\VirtualDrives\D\Programme\cassandra\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
was not released before the reference was garbage collected
ERROR [Reference-Reaper:1] 2015-06-24 14:12:02,603 Ref.java:179 - LEAK DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@13e1f0a) to class org.apache.cassandra.io.util.PoolingSegmentedFile$Cleanup@23584166:C:\VirtualDrives\D\Programme\cassandra\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866-Index.db
was not released before the reference was garbage collected
{code}

> FSReadError and LEAK DETECTED after upgrading
> ---------------------------------------------
>
>                 Key: CASSANDRA-9686
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9686
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>         Environment: Windows-7-32 bit, 3.2GB RAM, Java 1.7.0_55
>            Reporter: Andreas Schnitzerling
>            Assignee: Stefania
>             Fix For: 2.2.x
>
>         Attachments: cassandra.bat, cassandra.yaml, compactions_in_progress.zip, sstable_activity.zip,
system.log
>
>
> After upgrading one of 15 nodes from 2.1.7 to 2.2.0-rc1 I get FSReadError and LEAK DETECTED
on start. Deleting the listed files, the failure goes away.
> {code:title=system.log}
> ERROR [SSTableBatchOpen:1] 2015-06-29 14:38:34,554 DebuggableThreadPoolExecutor.java:242
- Error in ThreadPoolExecutor
> org.apache.cassandra.io.FSReadError: java.io.IOException: Compressed file with 0 chunks
encountered: java.io.DataInputStream@1c42271
> 	at org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> 	at org.apache.cassandra.io.compress.CompressionMetadata.<init>(CompressionMetadata.java:117)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> 	at org.apache.cassandra.io.compress.CompressionMetadata.create(CompressionMetadata.java:86)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> 	at org.apache.cassandra.io.util.CompressedSegmentedFile$Builder.metadata(CompressedSegmentedFile.java:142)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> 	at org.apache.cassandra.io.util.CompressedPoolingSegmentedFile$Builder.complete(CompressedPoolingSegmentedFile.java:101)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> 	at org.apache.cassandra.io.util.SegmentedFile$Builder.complete(SegmentedFile.java:178)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> 	at org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:681)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> 	at org.apache.cassandra.io.sstable.format.SSTableReader.load(SSTableReader.java:644)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> 	at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:443)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> 	at org.apache.cassandra.io.sstable.format.SSTableReader.open(SSTableReader.java:350)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> 	at org.apache.cassandra.io.sstable.format.SSTableReader$4.run(SSTableReader.java:480)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source) ~[na:1.7.0_55]
> 	at java.util.concurrent.FutureTask.run(Unknown Source) ~[na:1.7.0_55]
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_55]
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_55]
> 	at java.lang.Thread.run(Unknown Source) [na:1.7.0_55]
> Caused by: java.io.IOException: Compressed file with 0 chunks encountered: java.io.DataInputStream@1c42271
> 	at org.apache.cassandra.io.compress.CompressionMetadata.readChunkOffsets(CompressionMetadata.java:174)
~[apache-cassandra-2.2.0-rc1.jar:2.2.0-rc1]
> 	... 15 common frames omitted
> ERROR [Reference-Reaper:1] 2015-06-29 14:38:34,734 Ref.java:189 - LEAK DETECTED: a reference
(org.apache.cassandra.utils.concurrent.Ref$State@3e547f) to class org.apache.cassandra.io.sstable.format.SSTableReader$InstanceTidier@1926439:D:\Programme\Cassandra\data\data\system\compactions_in_progress\system-compactions_in_progress-ka-6866
was not released before the reference was garbage collected
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message