cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tommi Koivula (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-2307) IndexOutOfBoundsException during compaction
Date Wed, 19 Dec 2012 09:01:16 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-2307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13535792#comment-13535792
] 

Tommi Koivula commented on CASSANDRA-2307:
------------------------------------------

We're getting exactly the same error when running a forced major compaction with Cassandra
1.1.0. One of the CFs is producing this error every time. Compaction works for other CFs.
There is only one node in this cluster.

Cassandra version: 1.1.0
Environment: Ubuntu 12.04, 64-bit Sun JVM 1.6.0_33-b04

Stack trace:

java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at com.profium.rdfcassandra.CassandraBridge.compact(Unknown Source)
        at com.profium.rdfcassandra.CassandraBridge.addColumnFamily(Unknown Source)
        at com.profium.rdfcassandra.CassandraBackedIntegerPersistence.<init>(Unknown
Source)
        at com.profium.sir.ctx.ReasonerModule.createIntegerPersistence(ReasonerModule.java:149)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at com.profium.ioc.exec.PiocBeanFactory.create(PiocBeanFactory.java:528)
        at com.profium.ioc.exec.PiocBeanFactory.getOrCreate(PiocBeanFactory.java:456)
        at com.profium.ioc.exec.PiocBeanDefinitionExecutor.executeCreate(PiocBeanDefinitionExecutor.java:32)
        at com.profium.ioc.ctx.ApplicationContextFactory.createApplicationContext(ApplicationContextFactory.java:186)
        at com.profium.ioc.ctx.ApplicationContextFactory.initApplicationContext(ApplicationContextFactory.java:382)
        at SIR.exec.SirImpl.doChangeMdsRunlevel(SirImpl.java:799)
        at SIR.exec.SirImpl.raiseToRunlevel(SirImpl.java:685)
        at com.profium.sir.exec.MdsMonitorThread$1.run(MdsMonitorThread.java:84)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.IndexOutOfBoundsException
        at java.nio.Buffer.checkIndex(Buffer.java:518)
        at java.nio.HeapByteBuffer.getInt(HeapByteBuffer.java:340)
        at org.apache.cassandra.db.DeletedColumn.getLocalDeletionTime(DeletedColumn.java:50)
        at org.apache.cassandra.db.Column.isMarkedForDelete(Column.java:110)
        at org.apache.cassandra.db.Column.reconcile(Column.java:207)
        at org.apache.cassandra.db.DeletedColumn.reconcile(DeletedColumn.java:58)
        at org.apache.cassandra.db.ArrayBackedSortedColumns.resolveAgainst(ArrayBackedSortedColumns.java:168)
        at org.apache.cassandra.db.ArrayBackedSortedColumns.addAllColumns(ArrayBackedSortedColumns.java:232)
        at org.apache.cassandra.db.AbstractThreadUnsafeSortedColumns.addAll(AbstractThreadUnsafeSortedColumns.java:98)
        at org.apache.cassandra.db.AbstractColumnContainer.addAll(AbstractColumnContainer.java:92)
        at org.apache.cassandra.db.AbstractColumnContainer.addAll(AbstractColumnContainer.java:97)
        at org.apache.cassandra.db.compaction.PrecompactedRow.merge(PrecompactedRow.java:126)
        at org.apache.cassandra.db.compaction.PrecompactedRow.<init>(PrecompactedRow.java:99)
        at org.apache.cassandra.db.compaction.CompactionController.getCompactedRow(CompactionController.java:145)
        at org.apache.cassandra.db.compaction.CompactionIterable$Reducer.getReduced(CompactionIterable.java:97)
        at org.apache.cassandra.db.compaction.CompactionIterable$Reducer.getReduced(CompactionIterable.java:82)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.consume(MergeIterator.java:118)
        at org.apache.cassandra.utils.MergeIterator$ManyToOne.computeNext(MergeIterator.java:101)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
        at com.google.common.collect.Iterators$7.computeNext(Iterators.java:614)
        at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:140)
        at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:135)
        at org.apache.cassandra.db.compaction.CompactionTask.execute(CompactionTask.java:177)
        at org.apache.cassandra.db.compaction.CompactionManager$7.runMayThrow(CompactionManager.java:360)
        at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:30)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
        ... 1 more

                
> IndexOutOfBoundsException during compaction
> -------------------------------------------
>
>                 Key: CASSANDRA-2307
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-2307
>             Project: Cassandra
>          Issue Type: Bug
>    Affects Versions: 0.7.2, 0.7.3
>         Environment: Debian 5.0.8, Linux 2.6.26-2-amd64, 4GB of ram assigned to Cassandra,
JRE 1.6.0_24
>            Reporter: Mike Nadeau
>
> We're getting an IndexOutOfBounds exception when compacting.
> Here's the detailed error we get on-screen when running "nodetool -h 10.3.133.10 compact":
> Error occured while compacting keyspace test
> java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException
> at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
> at java.util.concurrent.FutureTask.get(Unknown Source)
> at org.apache.cassandra.db.CompactionManager.performMajor(CompactionManager.java:186)
> at org.apache.cassandra.db.ColumnFamilyStore.forceMajorCompaction(ColumnFamilyStore.java:1678)
> at org.apache.cassandra.service.StorageService.forceTableCompaction(StorageService.java:1248)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
> at com.sun.jmx.mbeanserver.StandardMBeanIntrospector.invokeM2(Unknown Source)
> at com.sun.jmx.mbeanserver.MBeanIntrospector.invokeM(Unknown Source)
> at com.sun.jmx.mbeanserver.PerInterface.invoke(Unknown Source)
> at com.sun.jmx.mbeanserver.MBeanSupport.invoke(Unknown Source)
> at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.invoke(Unknown Source)
> at com.sun.jmx.mbeanserver.JmxMBeanServer.invoke(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.doOperation(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.access$200(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl$PrivilegedOperation.run(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.doPrivilegedOperation(Unknown Source)
> at javax.management.remote.rmi.RMIConnectionImpl.invoke(Unknown Source)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
> at java.lang.reflect.Method.invoke(Unknown Source)
> at sun.rmi.server.UnicastServerRef.dispatch(Unknown Source)
> at sun.rmi.transport.Transport$1.run(Unknown Source)
> at java.security.AccessController.doPrivileged(Native Method)
> at sun.rmi.transport.Transport.serviceCall(Unknown Source)
> at sun.rmi.transport.tcp.TCPTransport.handleMessages(Unknown Source)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run0(Unknown Source)
> at sun.rmi.transport.tcp.TCPTransport$ConnectionHandler.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> Caused by: java.lang.IndexOutOfBoundsException
> at java.nio.Buffer.checkIndex(Unknown Source)
> at java.nio.HeapByteBuffer.getInt(Unknown Source)
> at org.apache.cassandra.db.DeletedColumn.getLocalDeletionTime(DeletedColumn.java:57)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:822)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedColumnsOnly(ColumnFamilyStore.java:809)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:800)
> at org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:94)
> at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:139)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
> at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
> at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
> at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
> at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:427)
> at org.apache.cassandra.db.CompactionManager$3.call(CompactionManager.java:217)
> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> ... 3 more
> And here's the error I'm getting in my log file:
> ERROR [CompactionExecutor:1] 2011-03-09 19:16:52,299 AbstractCassandraDaemon.java (line
114) Fatal exception in thread Thread[CompactionExecutor:1,1,main]
> java.lang.IndexOutOfBoundsException
> at java.nio.Buffer.checkIndex(Unknown Source)
> at java.nio.HeapByteBuffer.getInt(Unknown Source)
> at org.apache.cassandra.db.DeletedColumn.getLocalDeletionTime(DeletedColumn.java:57)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedStandard(ColumnFamilyStore.java:822)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeletedColumnsOnly(ColumnFamilyStore.java:809)
> at org.apache.cassandra.db.ColumnFamilyStore.removeDeleted(ColumnFamilyStore.java:800)
> at org.apache.cassandra.io.PrecompactedRow.<init>(PrecompactedRow.java:94)
> at org.apache.cassandra.io.CompactionIterator.getCompactedRow(CompactionIterator.java:139)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:108)
> at org.apache.cassandra.io.CompactionIterator.getReduced(CompactionIterator.java:43)
> at org.apache.cassandra.utils.ReducingIterator.computeNext(ReducingIterator.java:73)
> at com.google.common.collect.AbstractIterator.tryToComputeNext(AbstractIterator.java:136)
> at com.google.common.collect.AbstractIterator.hasNext(AbstractIterator.java:131)
> at org.apache.commons.collections.iterators.FilterIterator.setNextObject(FilterIterator.java:183)
> at org.apache.commons.collections.iterators.FilterIterator.hasNext(FilterIterator.java:94)
> at org.apache.cassandra.db.CompactionManager.doCompaction(CompactionManager.java:427)
> at org.apache.cassandra.db.CompactionManager$3.call(CompactionManager.java:217)
> at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
> at java.util.concurrent.FutureTask.run(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
> We run Cassandra 0.7.3, but we had the issue with Cassandra 0.7.2 as well. We have 8
machines in the cluster, the error happens only on one machine, at least for now.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message