Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Mon, 20 May 2013 18:17:24 +0000 (UTC)
From: "Vijay (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12631679.1360536707066.348058.1369073844061@arcas>
In-Reply-To: <JIRA.12631679.1360536707066@arcas>
References: <JIRA.12631679.1360536707066@arcas>
Subject: [jira] [Comment Edited] (CASSANDRA-5239) Fully Aysnc Server
 Transport (StorageProxy Layer)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-5239?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13662182#comment-13662182 ] 

Vijay edited comment on CASSANDRA-5239 at 5/20/13 6:16 PM:
-----------------------------------------------------------

{quote}
What kind of slowness did you observed? 
{quote}

Before:
Averages from the middle 80% of values:
interval_op_rate          : 15788
interval_key_rate         : 15788
latency median            : 0.9
latency 95th percentile   : 1.7
latency 99.9th percentile : 1440.9
Total operation time      : 00:01:59
END

After:
Averages from the middle 80% of values:
interval_op_rate          : 14164
interval_key_rate         : 14164
latency median            : 0.7
latency 95th percentile   : 2.0
latency 99.9th percentile : 2804.3
Total operation time      : 00:02:11

PS: this was on my laptop and not on real hardware... I did that to do some sanity check.

{quote}
The other solution would be to use the stress tool of https://github.com/datastax/java-driver
{quote}
Tried, but at-least i was not able to make that work (I could have done something dumb :)).
                
      was (Author: vijay2win@yahoo.com):
    {quote}
What kind of slowness did you observed? 
{quote}

After:
Averages from the middle 80% of values:
interval_op_rate          : 15788
interval_key_rate         : 15788
latency median            : 0.9
latency 95th percentile   : 1.7
latency 99.9th percentile : 1440.9
Total operation time      : 00:01:59
END

Before:
Averages from the middle 80% of values:
interval_op_rate          : 19740
interval_key_rate         : 19740
latency median            : 1.2
latency 95th percentile   : 4.6
latency 99.9th percentile : 112.3
Total operation time      : 00:00:47
END

PS: this was on my laptop and not on real hardware... I did that to do some sanity check.

{quote}
The other solution would be to use the stress tool of https://github.com/datastax/java-driver
{quote}
Tried, but at-least i was not able to make that work (I could have done something dumb :)).

                  
> Fully Aysnc Server Transport (StorageProxy Layer)
> -------------------------------------------------
>
>                 Key: CASSANDRA-5239
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5239
>             Project: Cassandra
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 2.0
>            Reporter: Vijay
>            Assignee: Vijay
>            Priority: Minor
>             Fix For: 2.0
>
>
> Problem Statement: 
> Currently we have "rpc_min_threads, rpc_max_threads"/ "native_transport_min_threads/native_transport_max_threads" all of the threads in the TPE are blocking and takes resources, the threads are mostly sleeping. Increasing the Context switch costs.
> Details: 
> We should change StorageProxy methods to provide a callback which contains the location where the results has to be written. When the response arrive StorageProxy callback can write the results directly into the connection. Timeouts can be handled in the same way.
> Fixing Netty should be trivial with some refactor in the storage proxy (currently it is one method call for sending the request and waiting) we need callback.
> Fixing Thrift may be harder because thrift calls the method and expects a return value. We might need to write a custom Codec on Netty for thrift support, which can potentially do callbacks (A Custom codec may be similar to http://engineering.twitter.com/2011/04/twitter-search-is-now-3x-faster_1656.html but we dont know details about it). Another option is to update thrift to have a callback.
> FYI, The motivation for this ticket is from another project which i am working on with similar Proxy (blocking Netty transport) and making it Async gave us 2x throughput improvement.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira