cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ariel Weisberg (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-8692) Coalesce intra-cluster network messages
Date Fri, 06 Feb 2015 22:42:36 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-8692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14310079#comment-14310079
] 

Ariel Weisberg commented on CASSANDRA-8692:
-------------------------------------------

1-4 make sense. #5 you might be right I need to check. When I was doing the unit tests I had
a problem with a large sleep, but I think I fixed that after.

The timing of providing messages to the average doesn't matter since the timestamp is generated
by the submitter not the sending thread. It's already only a vague stab at how often messages
are arriving. Is assuming a uniform distribution safe? You could be tricked into thinking
coalescing makes sense and then not do it effectively. Maybe with a smaller time horizon it
would be safer?

It seems like trying to do better would help bring down average latency by 100, maybe 200,
microseconds when latency is 2+ milliseconds. Average latency is still competitive with no
coalescing. Coalescing is turned off when there is less load (and average latency is lower)
so you don't feel the pinch there either.

I am on the fence as to whether something smarter than a moving average counts as scope creep.
I see that as fodder for future iteration unless you want to make the case that there is a
scenario where the current implementation is not a safe default and will give people a bad
experience. Maybe that is something I need to test for.

> Coalesce intra-cluster network messages
> ---------------------------------------
>
>                 Key: CASSANDRA-8692
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-8692
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>            Reporter: Ariel Weisberg
>            Assignee: Ariel Weisberg
>             Fix For: 2.1.4
>
>         Attachments: batching-benchmark.png
>
>
> While researching CASSANDRA-8457 we found that it is effective and can be done without
introducing additional latency at low concurrency/throughput.
> The patch from that was used and found to be useful in a real life scenario so I propose
we implement this in 2.1 in addition to 3.0.
> The change set is a single file and is small enough to be reviewable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message