Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: local policy)
From: Jeff Charette <iomatix@yahoo.com>
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_8455EB7E-16F9-4DAB-947C-D0FF2852FCD2"
Message-Id: <52CD3AC6-F1A7-495A-936B-5F2E85AA0620@yahoo.com>
Mime-Version: 1.0 (Mac OS X Mail 6.2 \(1499\))
Subject: Re: Very slow queries to view in design doc
Date: Tue, 12 Mar 2013 10:22:49 -0400
References: <265B0514-CB04-4C5A-8824-77B3008F4D47@poplatek.fi>
 <CABvT1DH=g=MV-BzLMogVLastc=MJtLo6EbLt_mNQsKB=1ygLTw@mail.gmail.com>
 <60C7D1C6-DE78-4798-9E8F-AB03E2F25C7D@poplatek.fi>
 <CABvT1DG6QOyGqFBZD_RyrMvogczgnCxpw1OMsYUNRf7y3FVXNw@mail.gmail.com>
 <5BB0B367-A723-4DC1-8C55-29E14CCC990A@poplatek.fi>
To: user@couchdb.apache.org
In-Reply-To: <5BB0B367-A723-4DC1-8C55-29E14CCC990A@poplatek.fi>

--Apple-Mail=_8455EB7E-16F9-4DAB-947C-D0FF2852FCD2
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=iso-8859-1

Is there a timeout message?

Jeff Charette | Principal=20
We Are Charette
web / identity / packaging

m  415.298.2707
w  wearecharette.com
e   jeffrey@wearecharette.com

On Mar 12, 2013, at 5:50 AM, Sami Sierla <sami.sierla@poplatek.fi> =
wrote:

> Hi,
>=20
> Issue happened again today. I was able to check file descriptors and =
established tcp connections to CouchDB. file descriptors in use were =
about 80 so they are not hitting any limits. Number of connections to =
couch before and after reboot were about 130.
>=20
> Reduce function counts sums from document attributes - there are about =
one million documents in view.
>=20
> Here's the raw json of map / reduce function:
>=20
> "Summary": {
>         "map": "function(doc) { if (doc.objectType =3D=3D \"Document\" =
&& (doc.severity !=3D=3D 4)) { emit(doc.alertTime, {\"feeds\": =
doc.feeds, \"severity\": doc.severity}); } }",
>         "reduce": "function(keys, values, rereduce) {\n  var i, j, k, =
v, res, feeds;\n  res =3D {};\n  if (!rereduce) {\n    res[\"*,*\"] =3D =
values.length;\n    for (i =3D 0; i < values.length; i++) {\n      k =3D =
\"*,\"+values[i].severity;\n      res[k] =3D (res[k] || 0) + 1;\n      =
feeds =3D values[i].feeds;\n      for (j =3D 0; j < feeds.length; j++) =
{\n        k =3D feeds[j]+\",*\";\n        res[k] =3D (res[k] || 0) + =
1;\n        k =3D feeds[j]+\",\"+values[i].severity;\n        res[k] =3D =
(res[k] || 0) + 1;\n      }\n    }\n  } else {\n     for (i =3D 0; i < =
values.length; i++) {\n        v =3D values[i];\n        for (k in v) =
{\n           res[k] =3D (res[k] || 0) + v[k];\n        }\n     }\n  }\n =
 return res;\n}\n"
>      }
>=20
> View entries are like this:
> =
{"id":"ac:431590","key":"2010-11-01T00:30:02.000+0000","value":{"feeds":["=
BNK"],"severity":2}}
>=20
> Again, this works just fine after reboot (even with reduce) but seems =
to slow over time (about 24-48 hours) until it starts to timeout.
>=20
> -Sami
>=20
> On Mar 11, 2013, at 11:20 PM, Robert Newson <rnewson@apache.org> =
wrote:
>=20
>> Can you access /_log on the server? might have information.
>>=20
>> Does the custom reduce function actually reduce (that is, return a
>> much smaller value than the input 'values' array)?
>>=20
>> B.
>>=20
>> On 11 March 2013 03:27, Sami Sierla <sami.sierla@poplatek.fi> wrote:
>>> Hi,
>>>=20
>>> There is custom reduce function in this view but result is same even =
if reduce=3Dfalse. Could this have something to do with running out of =
file descriptors (I unfortunately don't have sufficient user rights to =
check this on server). Issue began after we added two databases =
(unrelated to this view) to CouchDB (both have continuous replication to =
standby server).
>>>=20
>>> Best,
>>> Sami
>>>=20
>>> On Mar 8, 2013, at 4:20 PM, Robert Newson <rnewson@apache.org> =
wrote:
>>>=20
>>>> Do you have a custom reduce function? Are the keys/values in that =
view
>>>> very large?
>>>>=20
>>>> On 8 March 2013 08:18, Sami Sierla <sami.sierla@poplatek.fi> wrote:
>>>>> Hi,
>>>>>=20
>>>>> We are having odd issue with CouchDB 1.2.0 view queries - queries =
to one particular view are very slow (tens of seconds with limit=3D1) =
but queries to other views in the same design doc perform as they =
should. Issue goes away if we restart CouchDB but comes back after few =
hours. Views are periodically refreshed so there are no indexing =
blocking the query. The setup we have consists of about ten CouchDB =
databases on active db server (RHEL 5) and second standby server with =
continuous CouchDB pull replication. Same view query works fine on the =
standby server that has no other processes running.
>>>>>=20
>>>>> Any ideas what could cause this?
>>>>>=20
>>>>> Best,
>>>>> Sami Sierla
>>>=20
>=20


--Apple-Mail=_8455EB7E-16F9-4DAB-947C-D0FF2852FCD2--