cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sébastien Giroux (JIRA) <j...@apache.org>
Subject [jira] Commented: (CASSANDRA-2240) nodetool scrub hangs or throws an exception
Date Sun, 27 Feb 2011 21:54:37 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13000048#comment-13000048
] 

Sébastien Giroux commented on CASSANDRA-2240:
---------------------------------------------

We are running into the same issue using #325 from Hudson. Exception everytime.

Everything started when we upgraded from 0.6.8 to 0.7.2. One of the node (17 nodes total)
started to be slow, we realized one CF wasn't compacting and throw exception (Error in ThreadPoolExecutor
java.io.IOError: org.apache.cassandra.db.ColumnSerializer$CorruptColumnException: invalid
column name length 0).

So I gave the new scrub function a try and but looks like it doesnt work. We really need to
have this fixed!

Thank you!

> nodetool scrub hangs or throws an exception
> -------------------------------------------
>
>                 Key: CASSANDRA-2240
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2240
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Tools
>    Affects Versions: 0.7.3
>         Environment: using build #314 from hudson
>            Reporter: Yaniv Kunda
>            Assignee: Jonathan Ellis
>             Fix For: 0.7.3
>
>         Attachments: exception2.txt, jstack1.txt, signatureBuckets-f-104.tar.gz, test-0.6.x-tables.tar.gz,
userChannelFilter-f-210.tar.gz
>
>
> trying to run nodetool scrub hung or (only happened one time) threw the following exception:
> Error occured while scrubbing keyspace <mykeyspace>
> java.util.concurrent.ExecutionException: java.lang.AssertionError
> at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:252)
> at java.util.concurrent.FutureTask.get(FutureTask.java:111)
> at org.apache.cassandra.db.CompactionManager.performScrub(CompactionManager.java:203)
> at org.apache.cassandra.db.ColumnFamilyStore.scrub(ColumnFamilyStore.java:934)
> at org.apache.cassandra.service.StorageService.scrub(StorageService.java:1247)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:111)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(StandardMBeanIntrospector.java:45)
> at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(MBeanIntrospector.java:226)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(PerInterface.java:138)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(MBeanSupport.java:251)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(DefaultMBeanServerInterceptor.java:857)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(JmxMBeanServer.java:795)
> at javax.management.remote.rmi.RMIConnectionImpl.doOperation(RMIConnectionImpl.java:1450)
> at javax.management.remote.rmi.RMIConnectionImpl.access$200(RMIConnectionImpl.java:90)
> at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(RMIConnectionImpl.java:1285)
> at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(RMIConnectionImpl.java:1383)
> at javax.management.remote.rmi.RMIConnectionImpl.invoke(RMIConnectionImpl.java:807)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:616)
> at sun.rmi.server.UnicastServerRef.dispatch(UnicastServerRef.java:322)
> at sun.rmi.transport.Transport$1.run(Transport.java:177)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Transport.java:173)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(TCPTransport.java:553)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(TCPTransport.java:808)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(TCPTransport.java:667)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:636)
> Caused by: java.lang.AssertionError
> at org.apache.cassandra.dht.RandomPartitioner.convertFromDiskFormat(RandomPartitioner.java:62)
> at org.apache.cassandra.io.sstable.SSTableReader.decodeKey(SSTableReader.java:627)
> at org.apache.cassandra.db.CompactionManager.doScrub(CompactionManager.java:541)
> at org.apache.cassandra.db.CompactionManager.access$600(CompactionManager.java:55)
> at org.apache.cassandra.db.CompactionManager$3.call(CompactionManager.java:194)
> at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> ... 3 more

-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

       

Mime
View raw message