incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Ralev <vladimir.ra...@gmail.com>
Subject Re: Burst PUT updates
Date Fri, 27 Dec 2013 21:15:46 GMT
The client is not a problem. I observed the traffic with wireshark and the
requests make it while the the responses are getting slower and slower.

Today I observed that the server is busy with indexer tasks (10s of these),
but they only show up in http://HOST/_active_tasks not from futon. I will
think how can I confirm that this is the problem. This changes_done 385 and
432 don't make much sense to me since I it happens only after 50 or so PUTs.


   - {
      - pid: "<0.3769.9>",
      - changes_done: 385,
      - database: "account1",
      - design_document: "_design/account",
      - progress: 32,
      - started_on: 1388178498,
      - total_changes: 1194,
      - type: "indexer",
      - updated_on: 1388178500
      },
   -
   {
      - pid: "<0.3770.9>",
      - changes_done: 432,
      - database: "account2",
      - design_document: "_design/account",
      - progress: 36,
      - started_on: 1388178498,
      - total_changes: 1178,
      - type: "indexer",
      - updated_on: 1388178500
      },



On Fri, Dec 27, 2013 at 10:49 AM, Benoit Chesneau <bchesneau@gmail.com>wrote:

> On Fri, Dec 27, 2013 at 8:06 AM, Stanley Iriele <siriele2x3@gmail.com
> >wrote:
>
> > Even without bulk docs...that doesn't sound right..it sounds to me like
> the
> > library you're using is having a problem with its http client and the
> time
> > gap in closing a connection and opening a new one. Try looking at how
> many
> > open connections your app has when it starts to slow down.
> >
> > I'm guessing in the loop it tries to reuse the same TCP connection but it
> > can't so it just creates a new one..? Then slowly but surely your
> > application begins to die...I would override the TCP dial in the function
> > to count the number of created connections or use lsof.
> >
> > This is often why I stay away from SDK s for couchdb... They often limit
> > you more than they enable you... Your driver should be the http client
> > itself not some library that wraps making the calls for you.. In my
> opinion
> > Anyways 😀
> >
>
> The problem could also be be that  no framework here is used but a client.
> Couchdbkit for example is reusing connections. These frameworks aren't only
> "wrapping" the API call, they do much. At least for some.
>
> Anyway the problem can indeed be that either the client or couchdb run out
> of free fds. Then the system is waiting. Do you have any log in couchdb
> that can help? Or from your client?
>
> What is the limit of fds on your system ? (ulimit -n could help you on
> POSIX systems) ?
>
> - benoit
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message