cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andy Tolbert (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-9558) Cassandra-stress regression in 2.2
Date Sat, 06 Jun 2015 16:28:01 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-9558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andy Tolbert updated CASSANDRA-9558:
------------------------------------
    Attachment: atolber-trunk-driver-coalescing-disabled.txt

{quote}This logic makes sense on the server-side, since there is likely more useful work for
the server to do in the meantime, but on the client we're just delaying the server from getting
started on processing our requests. We should be batching, but not delaying. i.e. flushing
as soon as we have written all pending messages.{quote}

One motivation for coalescing/batching on the driver side is that in Netty 4.0 [channel.write
no longer implicitly flushes](http://netty.io/wiki/new-and-noteworthy-in-4.0.html#wiki-h4-20)
and it now must be explicit.  We found during testing that if you explicitly flush outside
of the netty event loop there is sizable performance cost.  I added a row to my table in the
previous comment showing the op rate degrading with coalescing disabled via -Dcom.datastax.driver.DISABLE_COALESCING=true
(also see attached [^atolber-trunk-driver-coalescing-disabled.txt]).  

That being said, if there is a more optimal way of doing this I think we should look into
it.  I think it would be interesting to see how things perform if the driver simply submits
a flush on the event loop and see what netty does for us (I'll try this out) compared to coalescing
or flushing outside the event loop.  If there is no perceived benefit of coalescing vs. submitting
a flush in the event loop we could consider changing the default behavior or removing the
functionality.

I think with my findings in my [previous comment|https://issues.apache.org/jira/browse/CASSANDRA-9558?focusedCommentId=14575616&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14575616],
this issue should be resolved by downgrading the protocol version to v2 (until the driver
is updated to support connection sizing with v3+) and setting core connections per host to
8. 

> Cassandra-stress regression in 2.2
> ----------------------------------
>
>                 Key: CASSANDRA-9558
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-9558
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Alan Boudreault
>            Priority: Blocker
>         Attachments: 2.1.log, 2.2.log, CASSANDRA-9558-2.patch, CASSANDRA-9558-ProtocolV2.patch,
atolber-CASSANDRA-9558-stress.tgz, atolber-trunk-driver-coalescing-disabled.txt, stress-2.1-java-driver-2.0.9.2.log,
stress-2.1-java-driver-2.2+PATCH.log, stress-2.1-java-driver-2.2.log, stress-2.2-java-driver-2.2+PATCH.log,
stress-2.2-java-driver-2.2.log
>
>
> We are seeing some regression in performance when using cassandra-stress 2.2. You can
see the difference at this url:
> http://riptano.github.io/cassandra_performance/graph_v5/graph.html?stats=stress_regression.json&metric=op_rate&operation=1_write&smoothing=1&show_aggregates=true&xmin=0&xmax=108.57&ymin=0&ymax=168147.1
> The cassandra version of the cluster doesn't seem to have any impact. 
> //cc [~tjake] [~benedict]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message