cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kévin LOVATO (JIRA) <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-6487) Log WARN on large batch sizes
Date Wed, 24 Sep 2014 09:08:34 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-6487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146103#comment-14146103
] 

Kévin LOVATO commented on CASSANDRA-6487:
-----------------------------------------

We experienced some problems when using (ridiculously) big batches here, Cassandra was throwing
odd exceptions:
{code}
java.lang.IndexOutOfBoundsException: Invalid combined index of 1565817280, maximum is 482109
        at org.jboss.netty.buffer.SlicedChannelBuffer.<init>(SlicedChannelBuffer.java:46)
        at org.jboss.netty.buffer.HeapChannelBuffer.slice(HeapChannelBuffer.java:201)
        at org.jboss.netty.buffer.AbstractChannelBuffer.readSlice(AbstractChannelBuffer.java:323)
        at org.apache.cassandra.transport.CBUtil.readValue(CBUtil.java:295)
        at org.apache.cassandra.transport.CBUtil.readValueList(CBUtil.java:340)
        at org.apache.cassandra.transport.messages.BatchMessage$1.decode(BatchMessage.java:62)
        at org.apache.cassandra.transport.messages.BatchMessage$1.decode(BatchMessage.java:43)
        at org.apache.cassandra.transport.Message$ProtocolDecoder.decode(Message.java:212)
        at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:66)
        at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:68)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:296)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
        at org.jboss.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:310)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:268)
        at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:255)
        at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:109)
        at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
        at org.jboss.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:90)
        at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
        at java.lang.Thread.run(Unknown Source)
{code}

We fixed our code so it behaved and stopped sending huge batches, but since it appears that
Cassandra can't handle too big batches, wouldn't it make more sense to simply refuse them
instead of warning / crashing ?

> Log WARN on large batch sizes
> -----------------------------
>
>                 Key: CASSANDRA-6487
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6487
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Patrick McFadin
>            Assignee: Lyuben Todorov
>            Priority: Minor
>             Fix For: 2.0.8, 2.1 beta2
>
>         Attachments: 6487-cassandra-2.0.patch, 6487-cassandra-2.0_v2.patch
>
>
> Large batches on a coordinator can cause a lot of node stress. I propose adding a WARN
log entry if batch sizes go beyond a configurable size. This will give more visibility to
operators on something that can happen on the developer side. 
> New yaml setting with 5k default.
> {{# Log WARN on any batch size exceeding this value. 5k by default.}}
> {{# Caution should be taken on increasing the size of this threshold as it can lead to
node instability.}}
> {{batch_size_warn_threshold: 5k}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message