incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dirk-Willem van Gulik <>
Subject Re: User, Erlang, Couchdb or kernel error ?
Date Sat, 31 Jan 2009 11:15:54 GMT
Paul Davis wrote:

> On Fri, Jan 30, 2009 at 12:23 PM, Dirk-Willem van Gulik
> <> wrote:
>  > Paul Davis wrote:
>  >
>  >> Taking a wild stab, are you running out of ports on the client side?
>  >> The 20-30 second wait would be when all your sockets are in TIME_WAIT
>  >> mode before closing?
>  >
>  > That seems to be fine -- BUT:

Looked into this a bit deeper.

While it was not holding us back - it does seem that either Erlang its 
socket lib or CouchDB behaves differently than, say, Apache when the 
connecting close happens (e.g. a telnet ends or a Connection: close is 
done). And there is something sub optimal. May be worth to file a JIRA 
to make sure this is dug into deeper.

>  >> You might check and see if your cilent lib is doing Keep-Alive or not.
>  > This gives me an opening - there is something odd with Keep-Alive. 
> Just to
>  > confirm - couchdb should do the right 1.1 dance, understand 
> Keep-Alive and
>  > allow for (anyish) number of subsequent requests ?

> I'm not sure on the exact specifics of Keep-Alive, but I think
> generally connections are created with a maximum-number of requests
> (something like 300) before the socket is closed. Not sure if this is
> a client or server setting.

After tweaking* both the python and perl libs a bit as to do keep alive 
differently; our 'caboom' problem is solved. As a raw telnet/curl are 
doing the 'right' thing - I suspect it is both these libraries which are 
at fault rather than anything else. I'll investigate more once we got 
through the various tests; but basically we're past the main hurdle and 
are now seeing 1500 req/sec on read and low 100's on writes; maxing out 
on disks in a fairly mundane setting (8 writers, 128 readers, few 
million of 6-9k byte records) for hours on stretch.

Thanks for the hint !


*: well, ok, cutting out all logic and hardcoding it.
This e-mail (and any attachments) is confidential and may contain personal views which are
not the views of the BBC unless specifically stated.
If you have received it in error, please delete it from your system.
Do not use, copy or disclose the information in any way nor act in reliance on it and notify
the sender immediately.
Please note that the BBC monitors e-mails sent or received.
Further communication will signify your consent to this.

View raw message