From dev-return-11613-apmail-couchdb-dev-archive=couchdb.apache.org@couchdb.apache.org Mon Aug 30 16:35:52 2010 Return-Path: Delivered-To: apmail-couchdb-dev-archive@www.apache.org Received: (qmail 38150 invoked from network); 30 Aug 2010 16:35:52 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 30 Aug 2010 16:35:52 -0000 Received: (qmail 25848 invoked by uid 500); 30 Aug 2010 16:35:51 -0000 Delivered-To: apmail-couchdb-dev-archive@couchdb.apache.org Received: (qmail 25791 invoked by uid 500); 30 Aug 2010 16:35:51 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 25775 invoked by uid 99); 30 Aug 2010 16:35:50 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Aug 2010 16:35:50 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of norman.barker@gmail.com designates 209.85.215.180 as permitted sender) Received: from [209.85.215.180] (HELO mail-ey0-f180.google.com) (209.85.215.180) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Aug 2010 16:35:43 +0000 Received: by eya25 with SMTP id 25so4096115eya.11 for ; Mon, 30 Aug 2010 09:35:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=qOzJ/H65ec79NlVB6/1x2uu6+Ik1RrtQO1uuuNoasco=; b=YfCCstEhpHGATIDLL1+I1eg4i0Mpxb67o/ptoN932RWh5rHb3lJPQu9A/YtHf6z9Mx z7Zui71sRWqzCn8GvXaz7+PRos7RIob5BWZ8yEzodx26P8z8rsX696FY2ZTSr377ISbX yGzsH1g3n7HuXcYIQ1eSJP7BeMSdcmy/uS8UM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=YgusYemWzcDaGN2PiNHPux3aom113uqbSCOiQG96DnTXYiqKh0d2UIFVkXssqIFX9u LbhDWVrgiin9phHcFDrc2Mvt7jU8gzB10i+HjiYbTKHMk0NPuvF3HIbMimYBXd/WiTd1 GbL8UwZiL4VksjpD2uaEOA/bMTEPk2jfyJ+YQ= MIME-Version: 1.0 Received: by 10.216.89.149 with SMTP id c21mr5190291wef.73.1283186122932; Mon, 30 Aug 2010 09:35:22 -0700 (PDT) Received: by 10.216.169.146 with HTTP; Mon, 30 Aug 2010 09:35:22 -0700 (PDT) In-Reply-To: References: <96CEF573-64EE-44B2-AC41-2A5A312A141D@apache.org> <4835209A-F0C3-493E-8ED9-E58FEC4B1C55@apache.org> <4C6B3C3B.8020607@gmail.com> Date: Mon, 30 Aug 2010 10:35:22 -0600 Message-ID: Subject: Re: multiview on github From: Norman Barker To: dev@couchdb.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org Hi, are there any more comments on this, if not can you describe the process (in particular how to obtain a wiki and jira account for couchdb which I have been unable to do) and I will start documenting this so we can put this into the trunk. Bob, were you able to do any more testing with large views, are there any suggestions on how to speed up the document id inclusion test as described below? thanks, Norman On Mon, Aug 23, 2010 at 9:22 AM, Norman Barker wr= ote: > Bob, > > thanks for the feedback and for taking a look at the code. Guidelines > on when to use a supervisor within couchdb with a gen_server would be > appreciated, currently I have a supervisor and a gen_server, but if > couchdb has a supervision process I could remove that layer. > > I think plugins is a great idea, however intersection of views is such > as common request, perhaps there needs to plugin system and if a > plugin is rated enough it goes into trunk as a core feature. > > the four (or slightly more) summary is here > > http://github.com/normanb/couchdb/raw/trunk/src/couchdb/couch_query_ring.= erl > > % > % send an id from the start list to the next node in the ring, if the > id is in adjacent node then the this node sends to the next ring node > .... > % if the id gets all round the ring and back to the start node then is > has intersected all queries and should be included. The nodes in the > ring > % should be sorted in size from small to large for this to be effective > % > % In addition send the initial id list round in parallel > > it really needs some eyes from the core couchdb coders to see how to > speed up the inclusion testing, looping is bad even if it is done in > parallel. > > Multiview is usable, I am using it with some pretty big mega-views (as > per the raindrop) model, I am also available to add features to this > as this is core part of our work and we want to give it to couch as a > contribution. > > thanks, > > Norman > > On Mon, Aug 23, 2010 at 5:05 AM, Robert Dionne > wrote: >> Hi Norman, >> >> =A0I took a peek at multiview. I haven't followed this too closely on th= e mailing list but this is *view intersection*? Is there a 5 line summary o= f what this does somewhere? >> >> =A0I'm curious as to why the daemon needs to be a supervisor, most if no= t all of the other daemons are gen_servers. OTP allows this but I think thi= s is a good area where some CouchDB guidelines on plugins would apply. >> >> =A0It strikes me that views, the use of map/reduce, etc. are one of the = trickier aspects of using CouchDB, particularly for new users coming from t= he SQL world. People are also reporting issues with performance of views, I= guess often because reduce functions go out of control. >> >> =A0I think the project would be better served if features like this were= available as plugins. I would put GeoCouch in the same category. Its very = neat and timely (given everyone wants to know where everyone else is using = their telephone but without talking other than asynchronously), but a serve= r plugin architecture that would allow this to be done cleanly should come = first. >> >> =A0This is just my opinion. I'd love to see some of the project founders= and committers weigh in on this and set some direction. >> >> Best regards, >> >> Bob >> >> >> >> >> >> On Aug 22, 2010, at 5:45 PM, Norman Barker wrote: >> >>> I would like to take this multiview code and have it added to trunk if >>> possible, what are the next steps? >>> >>> thanks, >>> >>> Norman >>> >>> On Wed, Aug 18, 2010 at 11:44 AM, Norman Barker wrote: >>>> I have made >>>> >>>> http://github.com/normanb/couchdb >>>> >>>> which is a fork of the latest couchdb trunk with the multiview code >>>> and tests added. >>>> >>>> If geocouch is available then it can still be used. >>>> >>>> There are a couple of questions about the multiview on the user /dev >>>> list so I will be adding some more test cases during today. >>>> >>>> thanks, >>>> >>>> Norman >>>> >>>> On Tue, Aug 17, 2010 at 9:23 PM, Norman Barker wrote: >>>>> this is possible, I forked geocouch since I use it, but I have alread= y >>>>> separated the geocouch dependencies from the trunk. >>>>> >>>>> I can do this tomorrow, certainly be interested in any feedback. >>>>> >>>>> thanks, >>>>> >>>>> Norman >>>>> >>>>> >>>>> >>>>> On Tue, Aug 17, 2010 at 7:49 PM, Volker Mische wrote: >>>>>> On 08/18/2010 03:26 AM, J Chris Anderson wrote: >>>>>>> >>>>>>> On Aug 16, 2010, at 4:38 PM, Norman Barker wrote: >>>>>>> >>>>>>>> Hi, >>>>>>>> >>>>>>>> I have made the changes as recommended, adding a test case >>>>>>>> multiview.js and also adding the userCtx to open the db. >>>>>>>> >>>>>>>> I have also forked geocouch and this is available here >>>>>>>> >>>>>>> >>>>>>> this patch seems important (especially as people are already asking= for >>>>>>> help using it on user@) >>>>>>> >>>>>>> to get it committed, it either must remove the dependency on GeoCou= ch, or >>>>>>> become part of CouchDB when (and if) GeoCouch becomes part of Couch= DB. >>>>>>> >>>>>>> Is it possible / useful to make a version that doesn't use GeoCouch= ? And >>>>>>> then to make the GeoCouch capabilities part GeoCouch for now? >>>>>>> >>>>>>> Chris >>>>>>> >>>>>> >>>>>> Hi Norman, >>>>>> >>>>>> if the patch is ready for trunk, I'd be happy to move the GeoCouch b= its to >>>>>> GeoCouch itself (as GeoCouch isn't ready for trunk yet). >>>>>> >>>>>> Lately I haven't been that responsive when it comes to GeoCouch, but= that >>>>>> will change (in about a month) after holidays and FOSS4G. >>>>>> >>>>>> Cheers, >>>>>> =A0Volker >>>>>> >>>>> >>>> >> >> >