incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From daniel williams <daniel.willi...@gmail.com>
Subject Re: couchdb .NET 4.0 client
Date Thu, 03 Nov 2011 20:32:59 GMT
Hi everyone,

To wrap up my initial query to the group.  After doing some more research
with my coworkers we were able to discover some additional obscure
information about the windows tcp stack.  I'll recap

   - our initial tests took code and ran it in .NET where the tcpdump
   showed the delay on the ack
   - we took the same code and compiled it in mono and ran it on mac and
   ran tcpdump and did not see the issue
   - we ran the same .exe that we ran on mac's version of mono and ran it
   through windows version of mono which showed the same delay.  This makes
   sense as there is still an OS call to get the tcp socket for the
   connectivity. This was run against many variations of windows (server 2003
   32 bit, windows 7 64 bit, 2008 64 bit) all displaying the same results
   roughly.  The ack after the 201 Created had a delay of roughly 200 ms.

We did the majority of our searches with queries similar to "windows
httpwebresponse getresponse slow".  A shot in the dark my boss searched
"tcp no delay windwows" and came across the following url:

http://lifeandcode.net/2009/05/reduce-game-network-latency-in-windows-7-or-vista/

After following the instructions we ran the same test and we were able to
product the following tcpdump with a vast improvement in speed:

4:12:46.*070*172 IP client.49761 > server.5984: P 20797:20962(165) ack
16561 win 63780
.....@.......oX..o..a.`|.....j.P..$7D..PUT
/test/testentity-f5719971-374c-4b61-93
4:12:46.071512 IP server.5984 > client.49761: P 16561:16586(25) ack 20962
win 65370
..A..@.......o...oX.`.a..j.|...P..Z;...HTTP/1.1 100 Continue


4:12:46.071545 IP client.49761 > server.5984: . ack 16586 win 63755
..(..@.......oX..o..a.`|.....j.P...6...
4:12:46.072854 IP client.49761 > server.5984: P 20962:21374(412) ack 16586
win 63755
.....@.......oX..o..a.`|.....j.P...8;..{"Foo1":"foo","Foo2":"foo","Foo3":"foo","F
4:12:46.077300 IP server.5984 > client.49761: P 16586:16911(325) ack 21374
win 64958
..m..@....v..o...oX.`.a..j.|..?P...>Q..HTTP/1.1 201 Created
erver: CouchDB/1.1.
4:12:46.077355 IP client.49761 > server.5984: . ack 16911 win 63430
..(..@.......oX..o..a.`|..?..lBP...6...
4:12:46.*077*687 IP server.5984 > client.49761: P 16911:17021(110) ack
21374 win 64958
.....@....L..o...oX.`.a..lB|..?P....c..{"ok":true,"id":"testentity-f5719971-374c-

With that said, this good be greatly beneficial to couchdb replication
schemes as well.  We've already seen marked improvement in couch processing
over http in testing scenarios.

Thanks.

dan


On Thu, Nov 3, 2011 at 10:56 AM, Dennis Redfield <
dennis.lee.redfield@gmail.com> wrote:

> wow.  I am going to work on this myself when time allows, very interesting.
>
> On Thursday, November 3, 2011, daniel williams <daniel.williams@gmail.com>
> wrote:
> > Oddly enough that is how I'm doing it.  Initially I thought that the
> > problem was with the Hammock library and began digging through the code
> and
> > noticed that they were creating the HttpWebRequest via the
> > WebRequest.Create.  In noticing this I thought there could be some
> > additional caching overhead in the Hammock library.  Pulling out exactly
> as
> > you have suggested produced the tcpdump I provided.  It is the Windows 64
> > bit / .NET 4.0 platform.  I ran the same tcpdump with Hammock and it has
> > produced the same results.  When I wrote the raw HTTP stream to a socket
> I
> > was able to do 4000 PUTs in 40s to a single instance of couchdb with the
> > 100-continue expect in line.
> >
> > I may simply rewrite GetById and Save to use a Socket instead of
> WebRequest
> > for the time being unless someone on this list knows of a better
> strategy.
> >
> > Thanks.
> >
> > dan
> >
> > On Thu, Nov 3, 2011 at 6:34 AM, Dennis Redfield <
> > dennis.lee.redfield@gmail.com> wrote:
> >
> >> I am running on Windows XP 32 bit and am making my own calls to couchdb
> >> using Framework 4.0 and they are very fast.  Nothing as slow as the
> number
> >> you are reporting.  I am thinking it problem is with the library not the
> >> underlying system - Why not write your own calls like:
> >>
> >>  HttpWebRequest myRequest =( HttpWebRequest ) WebRequest.Create(
> requestURL
> >> );
> >>               myRequest.MaximumAutomaticRedirections = _MaxRedirect;
> >>               myRequest.MaximumResponseHeadersLength = _MaxHeadersLen;
> >>               response = ( HttpWebResponse ) myRequest.GetResponse( );
> >>               Stream receiveStream=response.GetResponseStream( );
> >>               StreamReader readStream = new StreamReader( receiveStream,
> >> Encoding.UTF8 );//we shall see if this is right
> >>               return readStream.ReadToEnd( );
> >>
> >> Hope this helps.
> >>
> >> On Wednesday, November 2, 2011, daniel williams <
> daniel.williams@gmail.com
> >> >
> >> wrote:
> >> > Hi,
> >> >
> >> > I posted an question the other day in regard to latency leveraging the
> >> > Redbranch Hammock library on windows.  The codebase leverage the .NET
> 4.0
> >> > HttpWebRequest class which seems to have some serious issues in it's
> >> > communication with CouchDB.  I've noticed in a tcpdump on Mac with
> code
> >> > compiled in mono that a PUT/Save takes roughly 1~4ms total.  The same
> >> code
> >> > run on Mono or .NET takes about 200 ms and in analyzing the tcpdump on
> >> > windows (through mono or .NET making the execution) that after the
> Http
> >> > Continue 100 is received the .NET/Mono code on Windows 64 bit will
> wait
> >> > 200ms roughly before sending the ACK to receive the response payload.
> >>  See
> >> > below, I've changed the machine names:
> >> >
> >> > 10:03:04.552113 IP machine > server:5984: P 1:190(189) ack 1 win 256
> >> > E...X.@.......oX..o....`.eQ.Ug.bP...7\..PUT
> >> > /test/TestEntity-8cdddd58-1860-429e-90
> >> > 10:03:04.554113 IP server:5984 > machine: P 1:26(25) ack 190 win 65346
> >> > E..AI.@...zC..o...oX.`..Ug.b.eR.P..B....HTTP/1.1 100 Continue
> >> >
> >> >
> >> > 10:03:04.554220 IP machine > server:5984: P 190:602(412) ack 26 win
> 256
> >> > E...X.@
> >> >
> >>
> .......oX..o....`.eR.Ug.{P...8;..{"Foo1":"foo","Foo2":"foo","Foo3":"foo","F
> >> > 10:03:04.558000 IP server:5984 > machine: P 26:351(325) ack 602 win
> 64934
> >> > E..mI.@...y...o...oX.`..Ug.{.eTDP.......HTTP/1.1 201 Created
> >> > Server: CouchDB/1.1.
> >> > 10:03:04.751642 IP machine > server:5984: . ack 351 win 255
> >> > E..(X.@.......oX..o....`.eTDUg..P...6...
> >> > 10:03:04.752131 IP server:5984 > machine: P 351:461(110) ack 602 win
> >> 64934
> >> > E...I.@
> >> >
> >>
> ...y...o...oX.`..Ug...eTDP....Y..{"ok":true,"id":"TestEntity-8cdddd58-1860-
> >> >
> >> > In doing a test in isolation on Windows 64 bit and .NET 4.0 I wrote a
> >> > TcpClient implementation that meets HTTP standards and I was able to
> PUT
> >> > 4000 records successfully in less than 40 secs.  Does anyone out there
> >> know
> >> > of a more efficient library that can be used to communicate with the
> >> > CouchDB API on Windows 64 bit servers running .NET 4.0?  Looking at
> the
> >> > results of my tests run it would seem the most efficient to do
> straight
> >> > socket communication.
> >> >
> >> > Thanks.
> >> >
> >> > dan
> >> >
> >>
> >> --
> >> Trust No One
> >>
> >
> >
> >
> > --
> > -dan
> >
>
> --
> Trust No One
>



-- 
-dan

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message