Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of michael.greene@gmail.com
 designates 209.85.214.44 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=WE+xEMSVbSBqQejdir3Glq7AtnLqTGV+eQma5pS6b5CJSkdRELzVrcz4r1eOw3cVux
         VtDqrrEES+HImVpNwCmOCSAUG9j5ekx1HgeLHDwLbEBAA9CaNimkbACFl8/pZHevI1h+
         ZebsKTwmSM5Scz3H60IoHM7MbQvzXpaOclUGQ=
MIME-Version: 1.0
In-Reply-To: <AANLkTi=TZK0g+VYMRFZrtY9WBZ7cwf50aKbouNFmeTHm@mail.gmail.com>
References: <AANLkTi=-zAg=i6=zDJ6fN1_aEMnOvNryLdPbhh_pjBKS@mail.gmail.com>
	<AANLkTi=TZK0g+VYMRFZrtY9WBZ7cwf50aKbouNFmeTHm@mail.gmail.com>
Date: Fri, 17 Sep 2010 12:16:30 -0500
Message-ID: <AANLkTimHUudj8FHjPh5we0cXKpKasxhsaMwd2VNe4or6@mail.gmail.com>
Subject: Re: Network latency on Cassandra 0.7 (TFramedTransport)
From: Michael Greene <michael.greene@gmail.com>
To: user@cassandra.apache.org
Content-Type: multipart/alternative; boundary=00504502d81a495c71049077b9d3

--00504502d81a495c71049077b9d3
Content-Type: text/plain; charset=ISO-8859-1

This is the correct cause.  Reproducing your test gives 38-45ms in each of
10 runs.  If you run a profiler against it, you can see that the time is
entirely spent blocking on receive in TStreamTransport.Read.

Your test can be modified with the following line:
coreTransport.TcpClient.NoDelay = true;

After disabling Nagle's each test runs in < 1ms.

Michael

On Fri, Sep 17, 2010 at 9:57 AM, Peter Schuller <peter.schuller@infidyne.com
> wrote:

> > The latency is about 200 ms on every request when I'm connected to
> another
>
> My first thought here was that maybe you're seeing the effects of
> nagle[1] + delayed acks on the other side. On Unix, normally something
> like a thrift client would set TCP_NODELAY on its socket to avoid the
> problem. I'm not sure whether or not this is a reasonable hypothesis
> with C# + Thrift on Windows. Sorry, I don't know what the easy fix is
> either but perhaps it might help dig further.
>
> [1] http://en.wikipedia.org/wiki/Nagle's_algorithm
>
> --
> / Peter Schuller
>

--00504502d81a495c71049077b9d3
Content-Type: text/html; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

This is the correct cause. =A0Reproducing your test gives 38-45ms in each o=
f 10 runs. =A0If you run a profiler against it, you can see that the time i=
s entirely spent blocking on receive in TStreamTransport.Read.<div><br></di=
v>
<div>Your test can be modified with the following line:<div>coreTransport.T=
cpClient.NoDelay =3D true;</div><div><br>After disabling Nagle&#39;s each t=
est runs in &lt; 1ms.</div><div><br></div><div>Michael</div><div><br><div c=
lass=3D"gmail_quote">
On Fri, Sep 17, 2010 at 9:57 AM, Peter Schuller <span dir=3D"ltr">&lt;<a hr=
ef=3D"mailto:peter.schuller@infidyne.com">peter.schuller@infidyne.com</a>&g=
t;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0=
 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class=3D"im">&gt; The latency is about 200 ms on every request when I&=
#39;m connected to another<br>
<br>
</div>My first thought here was that maybe you&#39;re seeing the effects of=
<br>
nagle[1] + delayed acks on the other side. On Unix, normally something<br>
like a thrift client would set TCP_NODELAY on its socket to avoid the<br>
problem. I&#39;m not sure whether or not this is a reasonable hypothesis<br=
>
with C# + Thrift on Windows. Sorry, I don&#39;t know what the easy fix is<b=
r>
either but perhaps it might help dig further.<br>
<br>
[1] <a href=3D"http://en.wikipedia.org/wiki/Nagle&#39;s_algorithm" target=
=3D"_blank">http://en.wikipedia.org/wiki/Nagle&#39;s_algorithm</a><br>
<br>
--<br>
<font color=3D"#888888">/ Peter Schuller<br>
</font></blockquote></div><br></div></div>

--00504502d81a495c71049077b9d3--