Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Date: Mon, 16 Apr 2012 13:41:08 -0700
From: James Marca <jmarca@translab.its.uci.edu>
To: user@couchdb.apache.org
Subject: Re: Reduce just N rows?
Message-ID: <20120416204108.GE29414@translab.its.uci.edu>
Mail-Followup-To: user@couchdb.apache.org
References: 
 <CABzyt1G-19VR-To-ph=RqjYTUN0xZEFGs3TzJU8bO_ExsJeqtw@mail.gmail.com>
 <20120415061323.GD8135@translab.its.uci.edu>
 <CABzyt1HB_VD8n31qCQuBdcS-h2DGANMqwx+KRO+dfqHJ8TUDSw@mail.gmail.com>
 <20120416192525.GC29414@translab.its.uci.edu>
 <CABzyt1FQc0wOj1LPFnhRB9v4Uc+p5yO1UdO4bc8KGm12fwDmvA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
	protocol="application/pgp-signature"; boundary="bi5JUZtvcfApsciF"
Content-Disposition: inline
In-Reply-To: 
 <CABzyt1FQc0wOj1LPFnhRB9v4Uc+p5yO1UdO4bc8KGm12fwDmvA@mail.gmail.com>
User-Agent: Mutt/1.4.1i

--bi5JUZtvcfApsciF
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Mon, Apr 16, 2012 at 11:12:04PM +0300, Alon Keren wrote:
> On 16 April 2012 22:25, James Marca <jmarca@translab.its.uci.edu> wrote:
>=20
> > On Sun, Apr 15, 2012 at 12:00:38PM +0300, Alon Keren wrote:
> > > On 15 April 2012 09:13, James Marca <jmarca@translab.its.uci.edu> wro=
te:
> > >
...
> >
> > Also, I really don't think there is any load at all on the CPU with
> > this approach.  Or to be more accurate, no more than any active
> > database processing a view.  Again apologies for stating the obvious,
> > but CouchDB does incremenal updates of views, so if you keep adding
> > data, it only processes the new data.  Once you have processed the
> > data into a view, querying it (without reduce) takes almost no CPU.
> > Reducing it can be expensive if you do something in JavaScript, but
> > isn't as expensive if you stick with the built in native Erlang reduce
> > functions (sum, count, etc).
> >
>=20
> Reduces in couchdb should be incremental, unlike when doing them outside =
of
> couch.
>=20

Just to clarify what I mean, tes the reduces are incremental, but what
I mean by expensive is the cost of serializing data to JSON,
processing javascript, return to Erlang.  Reducing will do that at
least once as far as I understand, and sometimes more if the query
falls across tree boundaries.  for you, ten documents/rows is probably
cheap.  For me, with 120 large docs per hour per detector, it gets
expensive.  But if you stick with _sum type functions, you don't pay
for the serialization to JSON/JavsScript and back again.

http://wiki.apache.org/couchdb/Performance#Erlang_implementations_of_common=
_JavaScript_functions
http://wiki.apache.org/couchdb/Built-In_Reduce_Functions

For your case, you might try the _stats function on that second page
Although that wiki page is old, it is still in the code:
src/couchdb/couch_query_servers.erl.

Also, I prefer _sum and _count over _stats because you can pass in lists of
numbers, not just single numbers.  My views dump arrays of numbers,
and _sum does the right thing and produces an array of sums.  Stats
does not produce an array of arrays, sadly.


James

--bi5JUZtvcfApsciF
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)

iD8DBQFPjIPk+t/6L/9qydcRAl1MAJ4jQ5cxudP0iKxJUYHkijV53GcvuQCeJ2+2
eFITbJyfojLEGMboME210sc=
=JKlq
-----END PGP SIGNATURE-----

--bi5JUZtvcfApsciF--

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.