cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Jirsa (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-12633) Assert Errors occurring with OutboundTcpConnection causes Cassandra to slowly stop functioning.
Date Thu, 20 Oct 2016 04:49:58 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-12633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jeff Jirsa updated CASSANDRA-12633:
-----------------------------------
    Component/s: Streaming and Messaging

> Assert Errors occurring with OutboundTcpConnection causes Cassandra to slowly stop functioning.
> -----------------------------------------------------------------------------------------------
>
>                 Key: CASSANDRA-12633
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12633
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Streaming and Messaging
>            Reporter: Jeff Beck
>             Fix For: 2.1.x
>
>         Attachments: 12633-2.1.txt, 12633-2.2-start.txt, 12633-2.2.txt
>
>
> Assert Errors occurring with OutboundTcpConnection causes Cassandra to slowly stop functioning.
> There is a possibility of assert errors in ByteBufferUtil.writeWithShortLength which
can cause a stop to processing. There is already good error handling it just needs to catch
Throwable instead of Exception  at https://github.com/apache/cassandra/blob/c26bd91852cbf19d7dba9f62078f5da31b04dbe0/src/java/org/apache/cassandra/net/OutboundTcpConnection.java#L239

> An example of when this happens:
> {code}
> ERROR [MessagingService-Outgoing-/10.0.19.152] 2016-05-03 21:43:54,638 CassandraDaemon.java:227
- Exception in thread Thread[MessagingService-Outgoing-/10.0.19.152,5,main]
> java.lang.AssertionError: 103265
>         at org.apache.cassandra.utils.ByteBufferUtil.writeWithShortLength(ByteBufferUtil.java:290)
~[apache-cassandra-2.1.12.jar:2.1.12]
>    at org.apache.cassandra.db.composites.AbstractCType$Serializer.serialize(AbstractCType.java:392)
~[apache-cassandra-2.1.12.jar:2.1.12]
>      at org.apache.cassandra.db.composites.AbstractCType$Serializer.serialize(AbstractCType.java:381)
~[apache-cassandra-2.1.12.jar:2.1.12]
>      at org.apache.cassandra.db.filter.ColumnSlice$Serializer.serialize(ColumnSlice.java:271)
~[apache-cassandra-2.1.12.jar:2.1.12]
>      at org.apache.cassandra.db.filter.ColumnSlice$Serializer.serialize(ColumnSlice.java:259)
~[apache-cassandra-2.1.12.jar:2.1.12]
>      at org.apache.cassandra.db.filter.SliceQueryFilter$Serializer.serialize(SliceQueryFilter.java:503)
~[apache-cassandra-2.1.12.jar:2.1.12]
>    at org.apache.cassandra.db.filter.SliceQueryFilter$Serializer.serialize(SliceQueryFilter.java:490)
~[apache-cassandra-2.1.12.jar:2.1.12]
>      at org.apache.cassandra.db.SliceFromReadCommandSerializer.serialize(SliceFromReadCommand.java:168)
~[apache-cassandra-2.1.12.jar:2.1.12]
>  at org.apache.cassandra.db.ReadCommandSerializer.serialize(ReadCommand.java:143) ~[apache-cassandra-2.1.12.jar:2.1.12]
>    at org.apache.cassandra.db.ReadCommandSerializer.serialize(ReadCommand.java:132) ~[apache-cassandra-2.1.12.jar:2.1.12]
>    at org.apache.cassandra.net.MessageOut.serialize(MessageOut.java:121) ~[apache-cassandra-2.1.12.jar:2.1.12]
>       at org.apache.cassandra.net.OutboundTcpConnection.writeInternal(OutboundTcpConnection.java:330)
~[apache-cassandra-2.1.12.jar:2.1.12]
>     at org.apache.cassandra.net.OutboundTcpConnection.writeConnected(OutboundTcpConnection.java:282)
~[apache-cassandra-2.1.12.jar:2.1.12]
>    at org.apache.cassandra.net.OutboundTcpConnection.run(OutboundTcpConnection.java:218)
~[apache-cassandra-2.1.12.jar:2.1.12]
> {code}
> The exact error happened when you sent a read request via cql for a clustering column
value with more than 64k chars much like CASSANDRA-8280 but on reads. That exact error case
is no longer reproducable in the latest versions of C* but in general it seems like it should
still handle the possible asserts.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message