cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Pavel Yaskevich (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-4718) More-efficient ExecutorService for improved throughput
Date Sat, 17 May 2014 10:59:16 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-4718?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14000739#comment-14000739
] 

Pavel Yaskevich commented on CASSANDRA-4718:
--------------------------------------------

[~benedict] Isn't CQL trying to be smart about request routing like Thrift? Anyhow, read concurrency
was default 128 (can you confirm [~jasobrown]?) and we do remove all of the files, drop cache,
restart, disable compaction to remove jitter as much as possible, so the only difference between
two runs is one has sep patch another doesn't, if there are slow reads that should be happening
in both runs because keys are read uniformly and although there is a big amount of sstables
in the system for every read there is only one hit, which pretty much simulates the behavior
of the systems where data accumulates over time. Also I can tell you that setup that we have
for I/O is able to handle mere 300GB without a problem.

> More-efficient ExecutorService for improved throughput
> ------------------------------------------------------
>
>                 Key: CASSANDRA-4718
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-4718
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Jonathan Ellis
>            Assignee: Benedict
>            Priority: Minor
>              Labels: performance
>             Fix For: 2.1.0
>
>         Attachments: 4718-v1.patch, PerThreadQueue.java, austin_diskbound_read.svg, aws.svg,
aws_read.svg, backpressure-stress.out.txt, baq vs trunk.png, belliotsmith_branches-stress.out.txt,
jason_read.svg, jason_read_latency.svg, jason_write.svg, op costs of various queues.ods, stress
op rate with various queues.ods, stress_2014May15.txt, stress_2014May16.txt, v1-stress.out
>
>
> Currently all our execution stages dequeue tasks one at a time.  This can result in contention
between producers and consumers (although we do our best to minimize this by using LinkedBlockingQueue).
> One approach to mitigating this would be to make consumer threads do more work in "bulk"
instead of just one task per dequeue.  (Producer threads tend to be single-task oriented by
nature, so I don't see an equivalent opportunity there.)
> BlockingQueue has a drainTo(collection, int) method that would be perfect for this. 
However, no ExecutorService in the jdk supports using drainTo, nor could I google one.
> What I would like to do here is create just such a beast and wire it into (at least)
the write and read stages.  (Other possible candidates for such an optimization, such as the
CommitLog and OutboundTCPConnection, are not ExecutorService-based and will need to be one-offs.)
> AbstractExecutorService may be useful.  The implementations of ICommitLogExecutorService
may also be useful. (Despite the name these are not actual ExecutorServices, although they
share the most important properties of one.)



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message