cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-5239) Asynchronous (non-blocking) StorageProxy
Date Wed, 25 Mar 2015 22:46:54 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14380940#comment-14380940
] 

Benedict edited comment on CASSANDRA-5239 at 3/25/15 10:46 PM:
---------------------------------------------------------------

bq. are you talking about something like streaming a large result set?

Right. But the definition of "large" is important. Really I'm talking about ensuring all queries
require a fixed number of resources (or thereabouts), and that returning the entire result
can be done without significant extra overhead nor breaking isolation, so that even relatively
modest resultsets can be "streamed".

bq. doesn't seem like it's much better/worse

Exactly my point: as such it doesn't seem to be an intervening step, so using it as one isn't
a good justification for this patch. Either way, this is IMO one of the more pressing concerns
to address in the near future, and my view is we should decide what our approximate end goal
looks like concretely, and base any intervening states on that.


was (Author: benedict):
bq. doesn't seem like it's much better/worse

Exactly my point: as such it doesn't seem to be an intervening step, so using it as one isn't
a good justification for this patch. Either way, this is IMO one of the more pressing concerns
to address in the near future, and my view is we should decide what our approximate end goal
looks like concretely, and base any intervening states on that.

> Asynchronous (non-blocking) StorageProxy
> ----------------------------------------
>
>                 Key: CASSANDRA-5239
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5239
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 2.0 beta 1
>            Reporter: Vijay
>              Labels: performance
>             Fix For: 3.0
>
>
> Problem Statement: 
> Currently we have "rpc_min_threads, rpc_max_threads"/ "native_transport_min_threads/native_transport_max_threads"
all of the threads in the TPE are blocking and takes resources, the threads are mostly sleeping.
Increasing the Context switch costs.
> Details: 
> We should change StorageProxy methods to provide a callback which contains the location
where the results has to be written. When the response arrive StorageProxy callback can write
the results directly into the connection. Timeouts can be handled in the same way.
> Fixing Netty should be trivial with some refactor in the storage proxy (currently it
is one method call for sending the request and waiting) we need callback.
> Fixing Thrift may be harder because thrift calls the method and expects a return value.
We might need to write a custom Codec on Netty for thrift support, which can potentially do
callbacks (A Custom codec may be similar to http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html
but we dont know details about it). Another option is to update thrift to have a callback.
> FYI, The motivation for this ticket is from another project which i am working on with
similar Proxy (blocking Netty transport) and making it Async gave us 2x throughput improvement.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message