cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andreas Schnitzerling (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-6462) cleanup ClassCastException
Date Mon, 09 Dec 2013 17:20:08 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13843336#comment-13843336
] 

Andreas Schnitzerling commented on CASSANDRA-6462:
--------------------------------------------------

Cleanup on that node not working anymore. There are already problems after restart of the
cassandra service. Any ideas how to fix? If u don't have idea as well I will try to repair,
but maybe cannot reproduce anymore.
{panel:title=system.log}
ERROR [CompactionExecutor:25] 2013-12-09 17:50:14,932 CassandraDaemon.java (line 187) Exception
in thread Thread[CompactionExecutor:25,1,RMI Runtime]
java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@1dbf287
rejected from org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor@10f8a54[Terminated,
pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 780]
	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.reject(Unknown Source)
	at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(Unknown Source)
	at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(Unknown Source)
	at java.util.concurrent.ScheduledThreadPoolExecutor.submit(Unknown Source)
	at org.apache.cassandra.io.sstable.SSTableDeletingTask.schedule(SSTableDeletingTask.java:66)
	at org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1105)
	at org.apache.cassandra.db.DataTracker.removeOldSSTablesSize(DataTracker.java:388)
	at org.apache.cassandra.db.DataTracker.postReplace(DataTracker.java:353)
	at org.apache.cassandra.db.DataTracker.replace(DataTracker.java:347)
	at org.apache.cassandra.db.DataTracker.replaceCompactedSSTables(DataTracker.java:252)
	at org.apache.cassandra.db.ColumnFamilyStore.replaceCompactedSSTables(ColumnFamilyStore.java:1078)
	at org.apache.cassandra.db.compaction.CompactionTask.replaceCompactedSSTables(CompactionTask.java:296)
	at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:242)
	at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
	at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:60)
	at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
	at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:197)
	at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
ERROR [ValidationExecutor:1] 2013-12-09 17:53:16,411 Validator.java (line 242) Failed creating
a merkle tree for [repair #65efe6e0-60f2-11e3-ac47-eb1c24a59bb8 on nieste/nfiles, (3837863087520363054,3847126934337600104]],
/10.9.9.240 (see log for details)
ERROR [ValidationExecutor:1] 2013-12-09 17:53:16,421 CassandraDaemon.java (line 187) Exception
in thread Thread[ValidationExecutor:1,1,main]
FSWriteError in D:\Programme\cassandra\data\nieste\nfiles\snapshots\65efe6e0-60f2-11e3-ac47-eb1c24a59bb8\nieste-nfiles-jb-19878-Index.db
	at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:120)
	at org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:382)
	at org.apache.cassandra.io.util.FileUtils.deleteRecursive(FileUtils.java:378)
	at org.apache.cassandra.db.Directories.clearSnapshot(Directories.java:416)
	at org.apache.cassandra.db.ColumnFamilyStore.clearSnapshot(ColumnFamilyStore.java:1801)
	at org.apache.cassandra.db.compaction.CompactionManager.doValidationCompaction(CompactionManager.java:810)
	at org.apache.cassandra.db.compaction.CompactionManager.access$600(CompactionManager.java:62)
	at org.apache.cassandra.db.compaction.CompactionManager$8.call(CompactionManager.java:397)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
Caused by: java.nio.file.FileSystemException: D:\Programme\cassandra\data\nieste\nfiles\snapshots\65efe6e0-60f2-11e3-ac47-eb1c24a59bb8\nieste-nfiles-jb-19878-Index.db:
Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet
wird.

	at sun.nio.fs.WindowsException.translateToIOException(Unknown Source)
	at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
	at sun.nio.fs.WindowsException.rethrowAsIOException(Unknown Source)
	at sun.nio.fs.WindowsFileSystemProvider.implDelete(Unknown Source)
	at sun.nio.fs.AbstractFileSystemProvider.delete(Unknown Source)
	at java.nio.file.Files.delete(Unknown Source)
	at org.apache.cassandra.io.util.FileUtils.deleteWithConfirm(FileUtils.java:116)
	... 11 more
ERROR [CompactionExecutor:3] 2013-12-09 17:54:16,810 CassandraDaemon.java (line 187) Exception
in thread Thread[CompactionExecutor:3,1,RMI Runtime]
java.lang.ClassCastException: org.apache.cassandra.io.sstable.SSTableReader$EmptyCompactionScanner
cannot be cast to org.apache.cassandra.io.sstable.SSTableScanner
	at org.apache.cassandra.db.compaction.CompactionManager.doCleanupCompaction(CompactionManager.java:563)
	at org.apache.cassandra.db.compaction.CompactionManager.access$400(CompactionManager.java:62)
	at org.apache.cassandra.db.compaction.CompactionManager$5.perform(CompactionManager.java:274)
	at org.apache.cassandra.db.compaction.CompactionManager$2.call(CompactionManager.java:222)
	at java.util.concurrent.FutureTask.run(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.lang.Thread.run(Unknown Source)
{panel}

> cleanup ClassCastException
> --------------------------
>
>                 Key: CASSANDRA-6462
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6462
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Tools
>         Environment: Windows 7 / Java 1.7.0.25
>            Reporter: Andreas Schnitzerling
>              Labels: cast, cleanup, compaction, concurrency, newbie, nodetool, remote
>             Fix For: 2.0.4
>
>
> I enlarged the cluseter from 4 to 8 nodes. During cleaning up the "old" nodes with "nodetool
cleanup" it breaks up with exception. I started cleanup from a different computer to manage
them sequentially.
> {panel:title=cmd.exe}
> Error occurred during cleanup
> java.util.concurrent.ExecutionException: java.lang.ClassCastException: org.apach
> e.cassandra.io.sstable.SSTableReader$EmptyCompactionScanner cannot be cast to or
> g.apache.cassandra.io.sstable.SSTableScanner
>         at java.util.concurrent.FutureTask.report(Unknown Source)
>         at java.util.concurrent.FutureTask.get(Unknown Source)
>         at org.apache.cassandra.db.compaction.CompactionManager.performAllSSTabl
> eOperation(CompactionManager.java:227)
>         at org.apache.cassandra.db.compaction.CompactionManager.performCleanup(C
> ompactionManager.java:265)
>         at org.apache.cassandra.db.ColumnFamilyStore.forceCleanup(ColumnFamilySt
> ore.java:1054)
>         at org.apache.cassandra.service.StorageService.forceKeyspaceCleanup(Stor
> ageService.java:2038)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>         at java.lang.reflect.Method.invoke(Unknown Source)
>         at sun.reflect.misc.Trampoline.invoke(Unknown Source)
>         at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>         at java.lang.reflect.Method.invoke(Unknown Source)
>         at sun.reflect.misc.MethodUtil.invoke(Unknown Source)
>         at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown So
> urce)
>         at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown So
> urce)
>         at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source)
>         at com.sun.jmx.mbeanserver.PerInterface.invoke(Unknown Source)
>         at com.sun.jmx.mbeanserver.MBeanSupport.invoke(Unknown Source)
>         at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(Unknown
> Source)
>         at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(Unknown Source)
>         at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Sou
> rce)
>         at javax.management.remote.rmi.RMIConnectionImpl.access$300(Unknown Sour
> ce)
>         at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run
> (Unknown Source)
>         at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(U
> nknown Source)
>         at javax.management.remote.rmi.RMIConnectionImpl.invoke(Unknown Source)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
>         at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
>         at java.lang.reflect.Method.invoke(Unknown Source)
>         at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
>         at sun.rmi.transport.Transport$1.run(Unknown Source)
>         at sun.rmi.transport.Transport$1.run(Unknown Source)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at sun.rmi.transport.Transport.serviceCall(Unknown Source)
>         at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
>         at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Sou
> rce)
>         at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Sour
> ce)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.ClassCastException: org.apache.cassandra.io.sstable.SSTable
> Reader$EmptyCompactionScanner cannot be cast to org.apache.cassandra.io.sstable.
> SSTableScanner
>         at org.apache.cassandra.db.compaction.CompactionManager.doCleanupCompact
> ion(CompactionManager.java:563)
>         at org.apache.cassandra.db.compaction.CompactionManager.access$400(Compa
> ctionManager.java:62)
>         at org.apache.cassandra.db.compaction.CompactionManager$5.perform(Compac
> tionManager.java:274)
>         at org.apache.cassandra.db.compaction.CompactionManager$2.call(Compactio
> nManager.java:222)
>         at java.util.concurrent.FutureTask.run(Unknown Source)
>         ... 3 more
> {panel}



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message