cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-7593) Errors when upgrading through several versions to 2.1
Date Fri, 25 Jul 2014 07:59:38 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-7593?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14074181#comment-14074181
] 

Benedict edited comment on CASSANDRA-7593 at 7/25/14 7:57 AM:
--------------------------------------------------------------

There's still something fishy going on, though, since the BoundedComposite should also be
empty (it should have a size of 0, just with an EOC component). That doesn't change the fact
that this bug could present itself in situations where it legitimately does have a different
clustering size to size, so needs its own fix. I would say the correct answer may be to have
a BoundedCellName, which implements CellName, so that the collection check knows to not treat
it as a general composite but to grab its clustering size.

Presumably independently there is also a reason that we've changed the behaviour from 2.0
-> 2.1, which might imply another bug if we don't translate the data as well. But it looks
to me like we could potentially have this situation arise without an upgrade from legitimate
user-provided data as well.

It also looks to me like we caught this by chance, and our 'upgrade through versions' tests
should include a wider variety of table types with a variety of data contents


was (Author: benedict):
There's still something fishy going on, though, since the BoundedComposite should also be
empty (it should have a size of 0, just with an EOC component). That doesn't change the fact
that this bug could present itself in situations where it legitimately does have a different
clustering size to size, so needs its own fix. I would say the correct answer may be to have
a BoundedCellName, which implements CellName, so that the collection check knows to not treat
it as a general composite but to grab its clustering size.

> Errors when upgrading through several versions to 2.1
> -----------------------------------------------------
>
>                 Key: CASSANDRA-7593
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7593
>             Project: Cassandra
>          Issue Type: Bug
>         Environment: java 1.7
>            Reporter: Russ Hatch
>            Assignee: Marcus Eriksson
>            Priority: Critical
>             Fix For: 2.1.0
>
>
> I'm seeing two different errors cropping up in the dtest which upgrades a cluster through
several versions.
> This is the more common error:
> {noformat}
> ERROR [GossipStage:10] 2014-07-22 13:14:30,028 CassandraDaemon.java:168 - Exception in
thread Thread[GossipStage:10,5,main]
> java.lang.AssertionError: null
>         at org.apache.cassandra.db.filter.SliceQueryFilter.shouldInclude(SliceQueryFilter.java:347)
~[main/:na]
>         at org.apache.cassandra.db.filter.QueryFilter.shouldInclude(QueryFilter.java:249)
~[main/:na]
>         at org.apache.cassandra.db.CollationController.collectAllData(CollationController.java:249)
~[main/:na]
>         at org.apache.cassandra.db.CollationController.getTopLevelColumns(CollationController.java:60)
~[main/:na]
>         at org.apache.cassandra.db.ColumnFamilyStore.getTopLevelColumns(ColumnFamilyStore.java:1873)
~[main/:na]
>         at org.apache.cassandra.db.ColumnFamilyStore.getColumnFamily(ColumnFamilyStore.java:1681)
~[main/:na]
>         at org.apache.cassandra.db.Keyspace.getRow(Keyspace.java:345) ~[main/:na]
>         at org.apache.cassandra.db.SliceFromReadCommand.getRow(SliceFromReadCommand.java:59)
~[main/:na]
>         at org.apache.cassandra.cql3.statements.SelectStatement.readLocally(SelectStatement.java:293)
~[main/:na]
>         at org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:302)
~[main/:na]
>         at org.apache.cassandra.cql3.statements.SelectStatement.executeInternal(SelectStatement.java:60)
~[main/:na]
>         at org.apache.cassandra.cql3.QueryProcessor.executeInternal(QueryProcessor.java:263)
~[main/:na]
>         at org.apache.cassandra.db.SystemKeyspace.getPreferredIP(SystemKeyspace.java:514)
~[main/:na]
>         at org.apache.cassandra.net.OutboundTcpConnectionPool.<init>(OutboundTcpConnectionPool.java:51)
~[main/:na]
>         at org.apache.cassandra.net.MessagingService.getConnectionPool(MessagingService.java:522)
~[main/:na]
>         at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:536)
~[main/:na]
>         at org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:689)
~[main/:na]
>         at org.apache.cassandra.net.MessagingService.sendReply(MessagingService.java:663)
~[main/:na]
>         at org.apache.cassandra.service.EchoVerbHandler.doVerb(EchoVerbHandler.java:40)
~[main/:na]
>         at org.apache.cassandra.net.MessageDeliveryTask.run(MessageDeliveryTask.java:62)
~[main/:na]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_60]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
~[na:1.7.0_60]
>         at java.lang.Thread.run(Thread.java:745) ~[na:1.7.0_60]
> {noformat}
> The same test sometimes fails with this exception instead:
> {noformat}
> ERROR [CompactionExecutor:4] 2014-07-22 16:18:21,008 CassandraDaemon.java:168 - Exception
in thread Thread[CompactionExecutor:4,1,RMI Runtime]
> java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask@7059d3e9
rejected from org.apache.cassandra.concurrent.DebuggableScheduledThreadPoolExecutor@108f1504[Terminated,
pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 95]
>         at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2048)
~[na:1.7.0_60]
>         at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:821)
~[na:1.7.0_60]
>         at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:325)
~[na:1.7.0_60]
>         at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:530)
~[na:1.7.0_60]
>         at java.util.concurrent.ScheduledThreadPoolExecutor.execute(ScheduledThreadPoolExecutor.java:619)
~[na:1.7.0_60]
>         at org.apache.cassandra.io.sstable.SSTableReader.scheduleTidy(SSTableReader.java:628)
~[main/:na]
>         at org.apache.cassandra.io.sstable.SSTableReader.tidy(SSTableReader.java:609)
~[main/:na]
>         at org.apache.cassandra.io.sstable.SSTableReader.releaseReference(SSTableReader.java:1596)
~[main/:na]
>         at org.apache.cassandra.db.DataTracker.replaceReaders(DataTracker.java:354) ~[main/:na]
>         at org.apache.cassandra.io.sstable.SSTableRewriter.abort(SSTableRewriter.java:200)
~[main/:na]
>         at org.apache.cassandra.db.compaction.CompactionTask.runWith(CompactionTask.java:192)
~[main/:na]
>         at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
~[main/:na]
>         at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28) ~[main/:na]
>         at org.apache.cassandra.db.compaction.CompactionTask.executeInternal(CompactionTask.java:74)
~[main/:na]
>         at org.apache.cassandra.db.compaction.AbstractCompactionTask.execute(AbstractCompactionTask.java:59)
~[main/:na]
>         at org.apache.cassandra.db.compaction.CompactionManager$BackgroundCompactionTask.run(CompactionManager.java:236)
~[main/:na]
>         at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) ~[na:1.7.0_60]
>         at java.util.concurrent.FutureTask.run(FutureTask.java:262) ~[na:1.7.0_60]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
~[na:1.7.0_60]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_60]
>         at java.lang.Thread.run(Thread.java:745) [na:1.7.0_60]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message