Return-Path: Delivered-To: apmail-couchdb-dev-archive@www.apache.org Received: (qmail 41336 invoked from network); 30 Aug 2010 16:49:34 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 30 Aug 2010 16:49:34 -0000 Received: (qmail 48360 invoked by uid 500); 30 Aug 2010 16:49:33 -0000 Delivered-To: apmail-couchdb-dev-archive@couchdb.apache.org Received: (qmail 48302 invoked by uid 500); 30 Aug 2010 16:49:33 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 48294 invoked by uid 99); 30 Aug 2010 16:49:33 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Aug 2010 16:49:33 +0000 X-ASF-Spam-Status: No, hits=0.0 required=10.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of robert.newson@gmail.com designates 209.85.212.180 as permitted sender) Received: from [209.85.212.180] (HELO mail-px0-f180.google.com) (209.85.212.180) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 30 Aug 2010 16:49:26 +0000 Received: by pxi7 with SMTP id 7so4296167pxi.11 for ; Mon, 30 Aug 2010 09:49:05 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type :content-transfer-encoding; bh=xWkpHCeTsFoWLDIHM+mWCl5k3t1jaDt8JXtNNwGVqGI=; b=hVpWJYEt3DokA9jLR7JSr3oAjwrDCmfaFxsDKr1x8dJm6Vvz0mDxYUoKGRg1nv3uJt EaRQEdxmeDnZPJjKpBMKm4gUnrnN2OhfJc8JlT0bxJjMfo3TZHDDreMYZ6mVCGPKkvUK TmZHO+buP6bRF66wC0EZQR4dXcV9Da/WI9txY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type:content-transfer-encoding; b=mxl9DsE5RGqM75wlaEagEvxrGNdx/WE0EshPU31s71BQtpSd4TDHBlrcovdp//Wfjl z13JZYVNg3JtnhRiTLKa+LFrOuf6/VDr5MnZ8Ip+Mhyls/XyT8TNts7KUt4xLBveCeaA BIHYXd4v+U+3j5SJvQD+/TYHTurUKIRfbRUXg= MIME-Version: 1.0 Received: by 10.114.133.15 with SMTP id g15mr5506061wad.72.1283186945155; Mon, 30 Aug 2010 09:49:05 -0700 (PDT) Received: by 10.229.20.130 with HTTP; Mon, 30 Aug 2010 09:49:04 -0700 (PDT) In-Reply-To: References: <96CEF573-64EE-44B2-AC41-2A5A312A141D@apache.org> <4835209A-F0C3-493E-8ED9-E58FEC4B1C55@apache.org> <4C6B3C3B.8020607@gmail.com> Date: Mon, 30 Aug 2010 17:49:04 +0100 Message-ID: Subject: Re: multiview on github From: Robert Newson To: dev@couchdb.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable X-Virus-Checked: Checked by ClamAV on apache.org I'm sorry, I've had no time to play with this at scale. On Mon, Aug 30, 2010 at 5:35 PM, Norman Barker wr= ote: > Hi, > > are there any more comments on this, if not can you describe the > process (in particular how to obtain a wiki and jira account for > couchdb which I have been unable to do) and I will start documenting > this so we can put this into the trunk. > > Bob, were you able to do any more testing with large views, are there > any suggestions on how to speed up the document id inclusion test as > described below? > > thanks, > > Norman > > On Mon, Aug 23, 2010 at 9:22 AM, Norman Barker = wrote: >> Bob, >> >> thanks for the feedback and for taking a look at the code. Guidelines >> on when to use a supervisor within couchdb with a gen_server would be >> appreciated, currently I have a supervisor and a gen_server, but if >> couchdb has a supervision process I could remove that layer. >> >> I think plugins is a great idea, however intersection of views is such >> as common request, perhaps there needs to plugin system and if a >> plugin is rated enough it goes into trunk as a core feature. >> >> the four (or slightly more) summary is here >> >> http://github.com/normanb/couchdb/raw/trunk/src/couchdb/couch_query_ring= .erl >> >> % >> % send an id from the start list to the next node in the ring, if the >> id is in adjacent node then the this node sends to the next ring node >> .... >> % if the id gets all round the ring and back to the start node then is >> has intersected all queries and should be included. The nodes in the >> ring >> % should be sorted in size from small to large for this to be effective >> % >> % In addition send the initial id list round in parallel >> >> it really needs some eyes from the core couchdb coders to see how to >> speed up the inclusion testing, looping is bad even if it is done in >> parallel. >> >> Multiview is usable, I am using it with some pretty big mega-views (as >> per the raindrop) model, I am also available to add features to this >> as this is core part of our work and we want to give it to couch as a >> contribution. >> >> thanks, >> >> Norman >> >> On Mon, Aug 23, 2010 at 5:05 AM, Robert Dionne >> wrote: >>> Hi Norman, >>> >>> =A0I took a peek at multiview. I haven't followed this too closely on t= he mailing list but this is *view intersection*? Is there a 5 line summary = of what this does somewhere? >>> >>> =A0I'm curious as to why the daemon needs to be a supervisor, most if n= ot all of the other daemons are gen_servers. OTP allows this but I think th= is is a good area where some CouchDB guidelines on plugins would apply. >>> >>> =A0It strikes me that views, the use of map/reduce, etc. are one of the= trickier aspects of using CouchDB, particularly for new users coming from = the SQL world. People are also reporting issues with performance of views, = I guess often because reduce functions go out of control. >>> >>> =A0I think the project would be better served if features like this wer= e available as plugins. I would put GeoCouch in the same category. Its very= neat and timely (given everyone wants to know where everyone else is using= their telephone but without talking other than asynchronously), but a serv= er plugin architecture that would allow this to be done cleanly should come= first. >>> >>> =A0This is just my opinion. I'd love to see some of the project founder= s and committers weigh in on this and set some direction. >>> >>> Best regards, >>> >>> Bob >>> >>> >>> >>> >>> >>> On Aug 22, 2010, at 5:45 PM, Norman Barker wrote: >>> >>>> I would like to take this multiview code and have it added to trunk if >>>> possible, what are the next steps? >>>> >>>> thanks, >>>> >>>> Norman >>>> >>>> On Wed, Aug 18, 2010 at 11:44 AM, Norman Barker wrote: >>>>> I have made >>>>> >>>>> http://github.com/normanb/couchdb >>>>> >>>>> which is a fork of the latest couchdb trunk with the multiview code >>>>> and tests added. >>>>> >>>>> If geocouch is available then it can still be used. >>>>> >>>>> There are a couple of questions about the multiview on the user /dev >>>>> list so I will be adding some more test cases during today. >>>>> >>>>> thanks, >>>>> >>>>> Norman >>>>> >>>>> On Tue, Aug 17, 2010 at 9:23 PM, Norman Barker wrote: >>>>>> this is possible, I forked geocouch since I use it, but I have alrea= dy >>>>>> separated the geocouch dependencies from the trunk. >>>>>> >>>>>> I can do this tomorrow, certainly be interested in any feedback. >>>>>> >>>>>> thanks, >>>>>> >>>>>> Norman >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Aug 17, 2010 at 7:49 PM, Volker Mische wrote: >>>>>>> On 08/18/2010 03:26 AM, J Chris Anderson wrote: >>>>>>>> >>>>>>>> On Aug 16, 2010, at 4:38 PM, Norman Barker wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> I have made the changes as recommended, adding a test case >>>>>>>>> multiview.js and also adding the userCtx to open the db. >>>>>>>>> >>>>>>>>> I have also forked geocouch and this is available here >>>>>>>>> >>>>>>>> >>>>>>>> this patch seems important (especially as people are already askin= g for >>>>>>>> help using it on user@) >>>>>>>> >>>>>>>> to get it committed, it either must remove the dependency on GeoCo= uch, or >>>>>>>> become part of CouchDB when (and if) GeoCouch becomes part of Couc= hDB. >>>>>>>> >>>>>>>> Is it possible / useful to make a version that doesn't use GeoCouc= h? And >>>>>>>> then to make the GeoCouch capabilities part GeoCouch for now? >>>>>>>> >>>>>>>> Chris >>>>>>>> >>>>>>> >>>>>>> Hi Norman, >>>>>>> >>>>>>> if the patch is ready for trunk, I'd be happy to move the GeoCouch = bits to >>>>>>> GeoCouch itself (as GeoCouch isn't ready for trunk yet). >>>>>>> >>>>>>> Lately I haven't been that responsive when it comes to GeoCouch, bu= t that >>>>>>> will change (in about a month) after holidays and FOSS4G. >>>>>>> >>>>>>> Cheers, >>>>>>> =A0Volker >>>>>>> >>>>>> >>>>> >>> >>> >> >