cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Piotr Kołaczkowski (JIRA) <>
Subject [jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
Date Thu, 18 Apr 2013 13:09:27 GMT


Piotr Kołaczkowski commented on CASSANDRA-4718:

Interesting thing, that after boosting the number of threads that invoke the process() method
from 1 to 16, Akka gets slower, while thread-pool per stage approach gets faster.

16 user threads invoking process(), 4 core i7 with HT (8-virtual cores):

2 stages: 
Sync:     28195 ns
Async:    26852 ns
Akka:     51651 ns

4 stages: 
Sync:     75295 ns
Async:    60381 ns
Akka:     85954 ns

8 stages: 
Sync:    176879 ns
Async:   124712 ns
Akka:    103073 ns

16 stages: 
Sync:    367728 ns
Async:   259715 ns
Akka:    146875 ns

top reports total ~780% CPU utilisation
  thread-pools:  ~60% system, ~40% user
  Akka:          ~15% system, ~85% user

I try to add Disruptor to the benchmark suite.
> More-efficient ExecutorService for improved throughput
> ------------------------------------------------------
>                 Key: CASSANDRA-4718
>                 URL:
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jonathan Ellis
>            Priority: Minor
>         Attachments: baq vs trunk.png,
> Currently all our execution stages dequeue tasks one at a time.  This can result in contention
between producers and consumers (although we do our best to minimize this by using LinkedBlockingQueue).
> One approach to mitigating this would be to make consumer threads do more work in "bulk"
instead of just one task per dequeue.  (Producer threads tend to be single-task oriented by
nature, so I don't see an equivalent opportunity there.)
> BlockingQueue has a drainTo(collection, int) method that would be perfect for this. 
However, no ExecutorService in the jdk supports using drainTo, nor could I google one.
> What I would like to do here is create just such a beast and wire it into (at least)
the write and read stages.  (Other possible candidates for such an optimization, such as the
CommitLog and OutboundTCPConnection, are not ExecutorService-based and will need to be one-offs.)
> AbstractExecutorService may be useful.  The implementations of ICommitLogExecutorService
may also be useful. (Despite the name these are not actual ExecutorServices, although they
share the most important properties of one.)

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message