cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sylvain Lebresne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CASSANDRA-5239) Fully Aysnc Server Transport (StorageProxy Layer)
Date Mon, 20 May 2013 17:05:17 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13662155#comment-13662155
] 

Sylvain Lebresne commented on CASSANDRA-5239:
---------------------------------------------

bq. we should modify tools/stress to add the native functionality

Well, there is a -b flag to stress to use the binary protocol. Not sure it's very good however
since it use a one-thread-per-connection paradigm which might not really make justice to the
binary protocol. The other solution would be to use the stress tool of https://github.com/datastax/java-driver,
though you'd want to apply the stress patch from CASSANDRA-5422 as otherwise the stress application
is a bottleneck.

bq. The thrift stress is slower than before...

What kind of slowness did you observed? (as said above, I haven't really tested it myself,
though I should :))
                
> Fully Aysnc Server Transport (StorageProxy Layer)
> -------------------------------------------------
>
>                 Key: CASSANDRA-5239
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5239
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 2.0
>            Reporter: Vijay
>            Assignee: Vijay
>            Priority: Minor
>             Fix For: 2.0
>
>
> Problem Statement: 
> Currently we have "rpc_min_threads, rpc_max_threads"/ "native_transport_min_threads/native_transport_max_threads"
all of the threads in the TPE are blocking and takes resources, the threads are mostly sleeping.
Increasing the Context switch costs.
> Details: 
> We should change StorageProxy methods to provide a callback which contains the location
where the results has to be written. When the response arrive StorageProxy callback can write
the results directly into the connection. Timeouts can be handled in the same way.
> Fixing Netty should be trivial with some refactor in the storage proxy (currently it
is one method call for sending the request and waiting) we need callback.
> Fixing Thrift may be harder because thrift calls the method and expects a return value.
We might need to write a custom Codec on Netty for thrift support, which can potentially do
callbacks (A Custom codec may be similar to http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html
but we dont know details about it). Another option is to update thrift to have a callback.
> FYI, The motivation for this ticket is from another project which i am working on with
similar Proxy (blocking Netty transport) and making it Async gave us 2x throughput improvement.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message