cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vijay (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
Date Thu, 06 Dec 2012 23:35:22 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13525991#comment-13525991
] 

Vijay edited comment on CASSANDRA-4718 at 12/6/12 11:34 PM:
------------------------------------------------------------

How about Per thread queue? was curious about this ticket and ran a test.
I have to also note that the stages content when we have a lot of CPU's (was testing with
64 core machines)
                
      was (Author: vijay2win@yahoo.com):
    How about Per thread queue? was curious about this ticket and ran a test and actually
it performs better than Executors.newFixedThreadPool...

with arg[0] = 100000
{code}
Making JIT happy
TimeTaken: 109
Real test starts!
running per thread exec Test!
TimeTaken: 8
running Plain old exec Test!
TimeTaken: 82

Test with multiple of 10!
TimeTaken: 152
running Plain old exec Test!
TimeTaken: 334

Test with multiple of 100!
TimeTaken: 48341
running Plain old exec Test!
TimeTaken: 52195
{code}

I have to also note that the stages content when we have a lot of CPU's (was testing with
64 core machines)
                  
> More-efficient ExecutorService for improved throughput
> ------------------------------------------------------
>
>                 Key: CASSANDRA-4718
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4718
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jonathan Ellis
>            Priority: Minor
>         Attachments: PerThreadQueue.java
>
>
> Currently all our execution stages dequeue tasks one at a time.  This can result in contention
between producers and consumers (although we do our best to minimize this by using LinkedBlockingQueue).
> One approach to mitigating this would be to make consumer threads do more work in "bulk"
instead of just one task per dequeue.  (Producer threads tend to be single-task oriented by
nature, so I don't see an equivalent opportunity there.)
> BlockingQueue has a drainTo(collection, int) method that would be perfect for this. 
However, no ExecutorService in the jdk supports using drainTo, nor could I google one.
> What I would like to do here is create just such a beast and wire it into (at least)
the write and read stages.  (Other possible candidates for such an optimization, such as the
CommitLog and OutboundTCPConnection, are not ExecutorService-based and will need to be one-offs.)
> AbstractExecutorService may be useful.  The implementations of ICommitLogExecutorService
may also be useful. (Despite the name these are not actual ExecutorServices, although they
share the most important properties of one.)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message