Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Thu, 18 Apr 2013 13:09:27 +0000 (UTC)
From: =?utf-8?Q?Piotr_Ko=C5=82aczkowski_=28JIRA=29?= <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12609097.1348596683790.185962.1366290567148@arcas>
In-Reply-To: <JIRA.12609097.1348596683790@arcas>
References: <JIRA.12609097.1348596683790@arcas>
Subject: [jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService
 for improved throughput
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=3Dcom.atlas=
sian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D=
13635141#comment-13635141 ]=20

Piotr Ko=C5=82aczkowski commented on CASSANDRA-4718:
-----------------------------------------------

Interesting thing, that after boosting the number of threads that invoke th=
e process() method from 1 to 16, Akka gets slower, while thread-pool per st=
age approach gets faster.

16 user threads invoking process(), 4 core i7 with HT (8-virtual cores):

{noformat}
2 stages:=20
Sync:     28195 ns
Async:    26852 ns
Akka:     51651 ns

4 stages:=20
Sync:     75295 ns
Async:    60381 ns
Akka:     85954 ns

8 stages:=20
Sync:    176879 ns
Async:   124712 ns
Akka:    103073 ns

16 stages:=20
Sync:    367728 ns
Async:   259715 ns
Akka:    146875 ns
{noformat}

top reports total ~780% CPU utilisation
  thread-pools:  ~60% system, ~40% user
  Akka:          ~15% system, ~85% user

I try to add Disruptor to the benchmark suite.
               =20
> More-efficient ExecutorService for improved throughput
> ------------------------------------------------------
>
>                 Key: CASSANDRA-4718
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4718
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jonathan Ellis
>            Priority: Minor
>         Attachments: baq vs trunk.png, PerThreadQueue.java
>
>
> Currently all our execution stages dequeue tasks one at a time.  This can=
 result in contention between producers and consumers (although we do our b=
est to minimize this by using LinkedBlockingQueue).
> One approach to mitigating this would be to make consumer threads do more=
 work in "bulk" instead of just one task per dequeue.  (Producer threads te=
nd to be single-task oriented by nature, so I don't see an equivalent oppor=
tunity there.)
> BlockingQueue has a drainTo(collection, int) method that would be perfect=
 for this.  However, no ExecutorService in the jdk supports using drainTo, =
nor could I google one.
> What I would like to do here is create just such a beast and wire it into=
 (at least) the write and read stages.  (Other possible candidates for such=
 an optimization, such as the CommitLog and OutboundTCPConnection, are not =
ExecutorService-based and will need to be one-offs.)
> AbstractExecutorService may be useful.  The implementations of ICommitLog=
ExecutorService may also be useful. (Despite the name these are not actual =
ExecutorServices, although they share the most important properties of one.=
)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrato=
rs
For more information on JIRA, see: http://www.atlassian.com/software/jira