Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of robert.newson@gmail.com
 designates 74.125.82.52 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=Eo2PRcUxTDTOssVUQY6SYB8rfvIibdz1bAdsFF7bi8siU1381I8XRdc6jMYYYqp0e+
         6UdlvryEwJp/ofa4a+r9pWzMbGABVyPCfxaCHjk8qm+6ASBAXtGTXNdwv9iVTMKg4m7c
         A08mMssY1UhTfbKMoEHGpqnBpSYp0wzJnmD60=
MIME-Version: 1.0
In-Reply-To: <a260fac31003240301o660b888dnae089dae31172813@mail.gmail.com>
References: <a260fac31003232244s7687e618s2005020c7de3a0aa@mail.gmail.com>
	 <ec7a93a1003240006h37314e15yb8be4742e8c94f72@mail.gmail.com>
	 <a260fac31003240101w258d8dd1j3eac0c340914fa5c@mail.gmail.com>
	 <ec7a93a1003240112x394c1324qc2a1bf414ac9f01@mail.gmail.com>
	 <a260fac31003240301o660b888dnae089dae31172813@mail.gmail.com>
Date: Wed, 24 Mar 2010 10:12:15 +0000
Message-ID: <46aeb24f1003240312o1303758erd8531446f0c87490@mail.gmail.com>
Subject: Re: perfomance?
From: Robert Newson <robert.newson@gmail.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1

iirc apachebench only speaks http/1.0 but uses a common violation to
support keep-alive. This likely confuses CouchDB which speaks
http/1.1.

keep-alive is also not the same as pipelining. keep-alive just reuses
connections, whereas HTTP pipelining sends multiple requests without
reading the responses. The responses are read as a batch later; this
allows you to largely circumvent the latency of a network roundtrip.

B.

P.S My local apachebench certainly uses http/1.0;

Benchmarking 127.0.0.1 (be patient)...INFO: POST header ==
---
GET / HTTP/1.0
Host: 127.0.0.1:5984
User-Agent: ApacheBench/2.3
Accept: */*


---
LOG: header received:
HTTP/1.0 200 OK
Server: CouchDB/0.11.0 (Erlang OTP/R13B)
Date: Wed, 24 Mar 2010 10:11:32 GMT
Content-Type: text/plain;charset=utf-8
Content-Length: 41
Cache-Control: must-revalidate


On Wed, Mar 24, 2010 at 10:01 AM, Vasili Batareykin <john2do@gmail.com> wrote:
> about keepalive:
> working:
> req:
> GET /uri HTTP/1.0
> Connection: Keep-Alive
> Host: somehost
> User-Agent: someagent
> Accept: */*
>
> reply:
> HTTP/1.1 200 OK
> Date: Wed, 24 Mar 2010 09:44:36 GMT
> Server: megaserver
> Last-Modified: Sun, 22 Jul 2007 17:00:00 GMT
> ETag: "4d6436-8cdd-435dd17316400"
> Accept-Ranges: bytes
> Content-Length: 36061
> Keep-Alive: timeout=15, max=100
> Connection: Keep-Alive
> Content-Type: somecontent/type
>
> mandory field in req is Connection: Keep-Alive.
> mandory(imho) fields in reply:
> Content-Length: 36061 (how much read ...)
> Keep-Alive: timeout=15, max=100 (timeouts)
> Connection: Keep-Alive (and this or "close" if not support this feature)
>
> but what we see in couchdb's reply to same req:
>
> HTTP/1.0 200 OK
> Server: CouchDB/0.10.0 (Erlang OTP/R13B)
> Date: Wed, 24 Mar 2010 09:37:27 GMT
> Content-Type: text/plain;charset=utf-8
> Content-Length: 41
> Cache-Control: must-revalidate
>
> Connection timeout n/a
> Connection type n/a
>
> i think RFC better source than my dump)
>
> in my case (replicated+vers file storage) erlang eats all CPU to process
> requests.
> mb there is better way to get files from couchdb? by version or by e-tag?
> or increase some kind buffers or memory pages?
> sorry i don't know internals.
>
> 2010/3/24 Randall Leeds <randall.leeds@gmail.com>
>
>> Yes, I do mean KeepAlive, sorry for the confusion.
>> CouchDB should support it. Can you show a dump of the headers received
>> by Couch somehow? Maybe there is something silly like an issue of case
>> with the headers.
>>
>> CouchDB cannot use sendfile (or some Erlang way to do the same)
>> because it puts bytes in the middle of the attachment on disk at
>> regular intervals. As I understand it, this is so that you can store
>> attachments that contain something like a database header without
>> breaking your database. Otherwise, storing something that looks like
>> the BTree header in an attachment could cause couch to open the
>> database incorrectly after an odd crash situation.
>>
>> This behavior maybe sounds very paranoid and strange, but one thing is
>> for sure: CouchDB is designed to be *robust*. Really, really, _really_
>> hard to lose your data.
>>
>> On Wed, Mar 24, 2010 at 01:01, Vasili Batareykin <john2do@gmail.com>
>> wrote:
>> > pipelining? you mean keepalive? ab hold test if you supply -k option (Use
>> > HTTP KeepAlive feature) seems that couchdb's httpd don't know about this)
>> > yes, throughput(in b/s) is better,but on localhost, if i use same test
>> with
>> > nginx i get around 1000 #/sec on 340k file. (344294.81 [Kbytes/sec]).
>> yes,
>> > nginx use sendfile for this operation. yes fs cache used too. but 70
>> #/sec
>> > with couchdb ...
>> >
>> > 2010/3/24 Randall Leeds <randall.leeds@gmail.com>
>> >
>> >> If you multiple (#/sec) by file size, are actually getting _better_
>> >> throughput with the larger files.
>> >> Do you know if ab command uses HTTP 1.1 pipelining? If not, HTTP
>> >> overhead would explain the extra time.
>> >>
>> >
>>
>