couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sami Sierla <sami.sie...@poplatek.fi>
Subject Re: Very slow queries to view in design doc
Date Tue, 12 Mar 2013 09:50:27 GMT
Hi,

Issue happened again today. I was able to check file descriptors and established tcp connections
to CouchDB. file descriptors in use were about 80 so they are not hitting any limits. Number
of connections to couch before and after reboot were about 130.

Reduce function counts sums from document attributes - there are about one million documents
in view.

Here's the raw json of map / reduce function:

 "Summary": {
         "map": "function(doc) { if (doc.objectType == \"Document\" && (doc.severity
!== 4)) { emit(doc.alertTime, {\"feeds\": doc.feeds, \"severity\": doc.severity}); } }",
         "reduce": "function(keys, values, rereduce) {\n  var i, j, k, v, res, feeds;\n  res
= {};\n  if (!rereduce) {\n    res[\"*,*\"] = values.length;\n    for (i = 0; i < values.length;
i++) {\n      k = \"*,\"+values[i].severity;\n      res[k] = (res[k] || 0) + 1;\n      feeds
= values[i].feeds;\n      for (j = 0; j < feeds.length; j++) {\n        k = feeds[j]+\",*\";\n
       res[k] = (res[k] || 0) + 1;\n        k = feeds[j]+\",\"+values[i].severity;\n     
  res[k] = (res[k] || 0) + 1;\n      }\n    }\n  } else {\n     for (i = 0; i < values.length;
i++) {\n        v = values[i];\n        for (k in v) {\n           res[k] = (res[k] || 0)
+ v[k];\n        }\n     }\n  }\n  return res;\n}\n"
      }

View entries are like this:
{"id":"ac:431590","key":"2010-11-01T00:30:02.000+0000","value":{"feeds":["BNK"],"severity":2}}

Again, this works just fine after reboot (even with reduce) but seems to slow over time (about
24-48 hours) until it starts to timeout.

-Sami

On Mar 11, 2013, at 11:20 PM, Robert Newson <rnewson@apache.org> wrote:

> Can you access /_log on the server? might have information.
> 
> Does the custom reduce function actually reduce (that is, return a
> much smaller value than the input 'values' array)?
> 
> B.
> 
> On 11 March 2013 03:27, Sami Sierla <sami.sierla@poplatek.fi> wrote:
>> Hi,
>> 
>> There is custom reduce function in this view but result is same even if reduce=false.
Could this have something to do with running out of file descriptors (I unfortunately don't
have sufficient user rights to check this on server). Issue began after we added two databases
(unrelated to this view) to CouchDB (both have continuous replication to standby server).
>> 
>> Best,
>> Sami
>> 
>> On Mar 8, 2013, at 4:20 PM, Robert Newson <rnewson@apache.org> wrote:
>> 
>>> Do you have a custom reduce function? Are the keys/values in that view
>>> very large?
>>> 
>>> On 8 March 2013 08:18, Sami Sierla <sami.sierla@poplatek.fi> wrote:
>>>> Hi,
>>>> 
>>>> We are having odd issue with CouchDB 1.2.0 view queries - queries to one
particular view are very slow (tens of seconds with limit=1) but queries to other views in
the same design doc perform as they should. Issue goes away if we restart CouchDB but comes
back after few hours. Views are periodically refreshed so there are no indexing blocking the
query. The setup we have consists of about ten CouchDB databases on active db server (RHEL
5) and second standby server with continuous CouchDB pull replication. Same view query works
fine on the standby server that has no other processes running.
>>>> 
>>>> Any ideas what could cause this?
>>>> 
>>>> Best,
>>>> Sami Sierla
>> 


Mime
View raw message