Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id EDE2B200B4F for ; Mon, 11 Jul 2016 17:15:13 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id EC78C160A7D; Mon, 11 Jul 2016 15:15:13 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 1B505160A5E for ; Mon, 11 Jul 2016 17:15:12 +0200 (CEST) Received: (qmail 87910 invoked by uid 500); 11 Jul 2016 15:15:11 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 87572 invoked by uid 99); 11 Jul 2016 15:15:11 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Jul 2016 15:15:11 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 3FB7D2C02AA for ; Mon, 11 Jul 2016 15:15:11 +0000 (UTC) Date: Mon, 11 Jul 2016 15:15:11 +0000 (UTC) From: "Anthony Cozzie (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Resolved] (CASSANDRA-11403) Serializer/Version mismatch during upgrades to C* 3.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 11 Jul 2016 15:15:14 -0000 [ https://issues.apache.org/jira/browse/CASSANDRA-11403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anthony Cozzie resolved CASSANDRA-11403. ---------------------------------------- Resolution: Duplicate > Serializer/Version mismatch during upgrades to C* 3.0 > ----------------------------------------------------- > > Key: CASSANDRA-11403 > URL: https://issues.apache.org/jira/browse/CASSANDRA-11403 > Project: Cassandra > Issue Type: Bug > Components: Streaming and Messaging > Reporter: Anthony Cozzie > > The problem line seems to be: > {code} > MessageOut message = readCommand.createMessage(MessagingService.instance().getVersion(endpoint)); > {code} > SinglePartitionReadCommand then picks the serializer based on the version: > {code} > return new MessageOut<>(MessagingService.Verb.READ, this, version < MessagingService.VERSION_30 ? legacyReadCommandSerializer : serializer); > {code} > However, OutboundTcpConnectionPool will test the payload size vs the version from its smallMessages connection: > {code} > return msg.payloadSize(smallMessages.getTargetVersion()) > LARGE_MESSAGE_THRESHOLD > {code} > Which is set when the connection/pool is created: > {code} > targetVersion = MessagingService.instance().getVersion(pool.endPoint()); > {code} > During an upgrade, this state can change between these two calls leading the 3.0 serializer being used on 2.x packets and the following stacktrace: > ERROR [OptionalTasks:1] 2016-03-07 19:53:06,445 CassandraDaemon.java:195 - Exception in thread Thread[OptionalTasks:1,5,main] > java.lang.AssertionError: null > at org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:632) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.db.ReadCommand$Serializer.serializedSize(ReadCommand.java:536) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.net.MessageOut.payloadSize(MessageOut.java:166) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.net.OutboundTcpConnectionPool.getConnection(OutboundTcpConnectionPool.java:72) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.net.MessagingService.getConnection(MessagingService.java:609) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.net.MessagingService.sendOneWay(MessagingService.java:758) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.net.MessagingService.sendRR(MessagingService.java:701) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.net.MessagingService.sendRRWithFailure(MessagingService.java:684) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.service.AbstractReadExecutor.makeRequests(AbstractReadExecutor.java:110) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.service.AbstractReadExecutor.makeDataRequests(AbstractReadExecutor.java:85) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.service.AbstractReadExecutor$NeverSpeculatingReadExecutor.executeAsync(AbstractReadExecutor.java:214) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.service.StorageProxy$SinglePartitionReadLifecycle.doInitialQueries(StorageProxy.java:1699) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.service.StorageProxy.fetchRows(StorageProxy.java:1654) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.service.StorageProxy.readRegular(StorageProxy.java:1601) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.service.StorageProxy.read(StorageProxy.java:1520) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.db.SinglePartitionReadCommand$Group.execute(SinglePartitionReadCommand.java:918) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:251) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:212) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.cql3.statements.SelectStatement.execute(SelectStatement.java:77) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.cql3.QueryProcessor.processStatement(QueryProcessor.java:206) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:237) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:252) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.cql3.QueryProcessor.process(QueryProcessor.java:247) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.auth.CassandraRoleManager.hasExistingRoles(CassandraRoleManager.java:354) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.auth.CassandraRoleManager.setupDefaultRole(CassandraRoleManager.java:330) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.auth.CassandraRoleManager.lambda$setup$148(CassandraRoleManager.java:165) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at org.apache.cassandra.auth.CassandraRoleManager$$Lambda$206/135350257.call(Unknown Source) ~[na:na] > at org.apache.cassandra.auth.CassandraRoleManager$4.run(CassandraRoleManager.java:379) ~[cassandra-all-3.0.3.903.jar:3.0.3.903] > at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_40] > at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_40] > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) ~[na:1.8.0_40] > at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) ~[na:1.8.0_40] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[na:1.8.0_40] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_40] > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_40] -- This message was sent by Atlassian JIRA (v6.3.4#6332)