couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Norman Barker <norman.bar...@gmail.com>
Subject querying multiple views
Date Thu, 08 Jul 2010 17:43:04 GMT
Hi,

I have been thinking about how to query multiple views at one time.

I have an erlang handler in couchdb that takes a http post containing
N view queries, each query contains a startkey and an endkey, I then
open up each view in parallel (using pmap) and accumulate the doc ids,
then I use the erlang sets module to get the unique values. All good
and looks pretty (and works), though it doesn't scale since I am
holding all the results on the server (potential memory overload!)
whereas I would like to stream the results to the client one by one.

I am thinking of doing the following but have some questions;

My first question is when I do

couch_view:fold(View, FoldlFun, FoldAccInit,
couch_httpd_view:make_key_options(Args)),

is there a way to call the _count reduce function in code to find the
number of rows in the slice between startkey and endkey?

If so, I would like to order all the views in the posted query
document by the result of _count from smallest to largest.

I would then fold over the smallest result view and pull each document
id (*) in turn.

With each document id I would then call each of the other views in
turn with their startkey and endkey and in addition include
startkey_docid and endkey_docid with the docid in * above, again
calling _count I can test for inclusion. If the doc id is in all views
then I will immediate stream this to the client.

Am I doing something stupid, is this optimal?

Any help with the programmatic _count call would be great.

thanks,

Norman

Mime
View raw message