incubator-couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Norman Barker <>
Subject Re: multiview on github
Date Fri, 06 Aug 2010 16:16:40 GMT

I will make those changes, it might be a couple of days as I am on travel.

I will clone geocouch as a starting point and add javascript tests as
you suggest.

I am bench marking with around 10000 docs and a couple of views
(including geocouch), the main issue with the folding over every
document to both find the number of docs in a view slice (between
startkey and endkey) and then again to test inclusion between views.

I am interested in taking this forward and appreciate any code feedback.



On Thu, Aug 5, 2010 at 6:26 PM, J Chris Anderson <> wrote:
> On Aug 5, 2010, at 4:32 PM, Jan Lehnardt wrote:
>> Hi Norman,
>> I still plan to look at your code, I know the others here
>> are fairly busy too, sorry for the review delay :)
> The code looks clean (but could use better comments about where in the flow each module
comes into play). I don't think we can guess about performance, instead we should benchmark
to make sure the ring approach is right.
> In CouchDB currently, it is possible to isolate requests against a single db. So you
use the security settings to prevent access to databases, etc. For this, using the userCtx
and switching away from couch_db:open_int() would make a big difference.
> This way people can query across dbs if they have read access to all of them.
> I think if you package this as a CouchDB fork on Github and add a few JavaScript tests,
it will be really useful for some folks. I like that it has geo support. Maybe we can target
it for inclusion in trunk just after GeoCouch goes in trunk (if Volker wants to put it in.)
> Also, for realtime hacking on this, you might find that the #couchdb IRC channel on Freenode
is a good place to solicit feedback. There are a lot of people on there doing Geo things that
would benefit from this. (They really wanna be able to intersect a Geo query with a Map Reduce
query, etc.)
> Chris
>> Cheers
>> Jan
>> --
>> On 5 Aug 2010, at 18:12, Norman Barker wrote:
>>> Hi,
>>> is there any interest in the multiview, I have fixed (3) below, but am
>>> still interested in approaches for (1) and (2).
>>> thanks,
>>> Norman
>>> On Fri, Jul 30, 2010 at 3:39 PM, Norman Barker <>
>>>> Hi,
>>>> a very initial version of the multiview is at
>>>> for discussion.
>>>> The views are intersected by using a ring of processes where each node
>>>> in the ring represents a view as follows;
>>>> % send an id from the start list to the next node in the ring, if the
>>>> id is in adjacent node then this node sends to the next ring node ....
>>>> % if the id gets all round the ring and back to the start node then it
>>>> has intersected all queries and should be included. The nodes in the
>>>> ring
>>>> % should be sorted in size from small to large for this to be effective
>>>> %
>>>> % In addition send the initial id list round in parallel
>>>> this is implemented in the couch_query_ring module.
>>>> I have a couple of questions
>>>> 1) in the module multiview, is there a quicker way to find the counts
>>>> from startkey to endkey rather than iterating?
>>>> 2) In the module couch_query_ring is there a quicker way to test for
>>>> inclusion rather than iterating?
>>>> 3) Finally, if I hit this concurrently I get an exception,
>>>> [error] [<0.201.0>] Uncaught error in HTTP request: {exit,
>>>>                                {noproc,
>>>>                                 {gen_server,call,
>>>> (so ignore my previous email, I am able to trap the msg)
>>>> I am going to look into (3) but if you have seen this before.
>>>> I am developing on windows, but also test on linux I will work on
>>>> getting a linux makefile, but the should be a start.
>>>> Any help and comments appreciated.
>>>> Norman

View raw message