From user-return-4079-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Wed Mar 18 19:00:34 2009 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 7838 invoked from network); 18 Mar 2009 19:00:33 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.2) by minotaur.apache.org with SMTP; 18 Mar 2009 19:00:33 -0000 Received: (qmail 7138 invoked by uid 500); 18 Mar 2009 19:00:29 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 7101 invoked by uid 500); 18 Mar 2009 19:00:29 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 7060 invoked by uid 99); 18 Mar 2009 19:00:29 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 Mar 2009 12:00:29 -0700 X-ASF-Spam-Status: No, hits=1.6 required=10.0 tests=RCVD_IN_DNSWL_LOW,RCVD_NUMERIC_HELO,SPF_HELO_PASS,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of gcdcu-couchdb-user@m.gmane.org designates 80.91.229.2 as permitted sender) Received: from [80.91.229.2] (HELO ciao.gmane.org) (80.91.229.2) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 18 Mar 2009 19:00:20 +0000 Received: from list by ciao.gmane.org with local (Exim 4.43) id 1Lk0zb-0002xH-5y for user@couchdb.apache.org; Wed, 18 Mar 2009 18:59:55 +0000 Received: from 96.49.96.189 ([96.49.96.189]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 18 Mar 2009 18:59:55 +0000 Received: from dan_the_man by 96.49.96.189 with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 18 Mar 2009 18:59:55 +0000 X-Injected-Via-Gmane: http://gmane.org/ To: user@couchdb.apache.org From: Daniel Friesen Subject: Re: Trying to see if CouchDB is right for our project... Date: Wed, 18 Mar 2009 10:46:23 -0700 Lines: 109 Message-ID: <49C1336F.1070603@telus.net> References: <64a10fff0903171343y21b66103nfd2581ae6057063d@mail.gmail.com> <64a10fff0903180744r65d10f96ra9c3722535ada8e0@mail.gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@ger.gmane.org X-Gmane-NNTP-Posting-Host: 96.49.96.189 User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.17) Gecko/20080914 Thunderbird/2.0.0.17 ThunderBrowse/3.2.1.9 Mnenhy/0.7.5.666 In-Reply-To: Sender: news X-Virus-Checked: Checked by ClamAV on apache.org I did have another idea to propose, haven't got a name for it yet. I just wanted to start with the simplest idea, this one required very little modification to CouchDB and would work in a pretty efficient way that in some small way could even be indexed for better performance (theoretically if the jsonParse being used supported the second callback function for mutating data, it might be possible to use that to index _document tags on save at the same time as parsing the json) Views are basically pieces of code for creating indexes so they only operate on a single document of data. So I was thinking of a method of querying for data server side and outputting it. _query if you will. ((pardon if I'm getting the general api path naming idea wrong)) /database/_query/queryname?param[key]=foo /database/document/_query/queryname?param[key]=foo db is the couch db, doc the doc if called in /database/document/... fashion, and params is the param data send in the url. /database/_query/jits?param[root]=id /database/id/_query/jits?param[root]=id query: function(db, doc, params) { var root = doc || db.get(params[root]); var hash = {}; var jitQue = [root]; var jit; while(jit = jitQue.shift()) { hash[jit] = db.get(jit); var c = hash[jit].children; for ( var i = 0, l = c.length; i On Wed, Mar 18, 2009 at 7:44 AM, Dean Landolt wrote: > >> On Wed, Mar 18, 2009 at 1:49 AM, Daniel Friesen wrote: >> >> >>> Ok, I started a document here: >>> http://wiki.apache.org/couchdb/Forward_document_references >>> >>> There is a little more of the document to write. And I don't get how to >>> format the code so it isn't messed up. >>> I'll ping the dev list when I get back. >>> >> Interesting ideas. One thing I'd add is that if this were implemented it >> would probably be best to make use of json referencing [1] which would allow >> you to make references by id or path. >> >> [1] http://www.json.com/2007/10/19/json-referencing-proposal-and-library/ >> >> > > I like the notion of applying this to views rather than documents, as > it is more general purpose. It's important that query techniques not > be tied to particular document schemas. Maybe the simplest way it > could work (start with simple first, then complete...) is to have the > view row (key, value) structure be: > > (key, [array of keys to drill into]) > > And then query with an option for how far to drill. This version of it > does not have room for a payload (other than the docids) so maybe it's > worth forcing the array into a field on the value, so > > (key, {"subkeys" : [array of keys to drill into], "foo" : "bar"}) > > Something like json-referencing could give us the ability to make the > value-schema flexible as well, but that's not core to the basic > question about recursive view traversal. > > The implementation is only successful if the drill-down can happen in > nearly constant memory space (so any gatherings would have to be > accomplished in a reduce-like phase, which we can consider out of > scope for now...). Anyone want to hazard a guess as to the big-O > performance of a beast like this? > > The basic memory bloat of buffering all the values can be avoided by > echoing them to the client and forgetting them. The only residual > bloat I see is the stack, which could become quite deep if depth is > unlimited. Another complication is circular references. > > If the implementation is correct I can see this as a good plugin for > CouchDB. Once it's written all you need is a bunch of people to use it > and like it, and it could be a solid candidate for inclusion in core. > > This thread dearly belongs on dev@ - cross posting this message, let's > try to keep subsequent replies on dev@. > > Cheers, > Chris > > >