incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Friesen <dan_the_...@telus.net>
Subject Re: Trying to see if CouchDB is right for our project...
Date Wed, 18 Mar 2009 17:46:23 GMT
I did have another idea to propose, haven't got a name for it yet. I 
just wanted to start with the simplest idea, this one required very 
little modification to CouchDB and would work in a pretty efficient way 
that in some small way could even be indexed for better performance 
(theoretically if the jsonParse being used supported the second callback 
function for mutating data, it might be possible to use that to index 
_document tags on save at the same time as parsing the json)

Views are basically pieces of code for creating indexes so they only 
operate on a single document of data. So I was thinking of a method of 
querying for data server side and outputting it. _query if you will.

((pardon if I'm getting the general api path naming idea wrong))
/database/_query/queryname?param[key]=foo
/database/document/_query/queryname?param[key]=foo

db is the couch db, doc the doc if called in /database/document/... 
fashion, and params is the param data send in the url.

/database/_query/jits?param[root]=id
/database/id/_query/jits?param[root]=id
query: function(db, doc, params) {
  var root = doc || db.get(params[root]);
  var hash = {};
  var jitQue = [root];
  var jit;
  while(jit = jitQue.shift()) {
    hash[jit] = db.get(jit);
    var c = hash[jit].children;
    for ( var i = 0, l = c.length; i<l; i++ ) {
      if( hash[c[i]] ) continue;
      jitQue.push(c[i]);
    }
  }
 
  return hash;
}

The db would allow for any type of access, be it directly to a document, 
or to a view. The ideal purpose would be only used when you have 
document ids you already know about (or view keys) and want to grab the 
data.

Another possible name might be _aggregate.

~Daniel Friesen (Dantman, Nadir-Seen-Fire)

Chris Anderson wrote:
> On Wed, Mar 18, 2009 at 7:44 AM, Dean Landolt <dean@deanlandolt.com> wrote:
>   
>> On Wed, Mar 18, 2009 at 1:49 AM, Daniel Friesen <dan_the_man@telus.net>wrote:
>>
>>     
>>> Ok, I started a document here:
>>> http://wiki.apache.org/couchdb/Forward_document_references
>>>
>>> There is a little more of the document to write. And I don't get how to
>>> format the code so it isn't messed up.
>>> I'll ping the dev list when I get back.
>>>       
>> Interesting ideas. One thing I'd add is that if this were implemented it
>> would probably be best to make use of json referencing [1] which would allow
>> you to make references by id or path.
>>
>> [1] http://www.json.com/2007/10/19/json-referencing-proposal-and-library/
>>
>>     
>
> I like the notion of applying this to views rather than documents, as
> it is more general purpose. It's important that query techniques not
> be tied to particular document schemas. Maybe the simplest way it
> could work (start with simple first, then complete...) is to have the
> view row (key, value) structure be:
>
> (key, [array of keys to drill into])
>
> And then query with an option for how far to drill. This version of it
> does not have room for a payload (other than the docids) so maybe it's
> worth forcing the array into a field on the value, so
>
> (key, {"subkeys" : [array of keys to drill into], "foo" : "bar"})
>
> Something like json-referencing could give us the ability to make the
> value-schema flexible as well, but that's not core to the basic
> question about recursive view traversal.
>
> The implementation is only successful if the drill-down can happen in
> nearly constant memory space (so any gatherings would have to be
> accomplished in a reduce-like phase, which we can consider out of
> scope for now...). Anyone want to hazard a guess as to the big-O
> performance of a beast like this?
>
> The basic memory bloat of buffering all the values can be avoided by
> echoing them to the client and forgetting them. The only residual
> bloat I see is the stack, which could become quite deep if depth is
> unlimited. Another complication is circular references.
>
> If the implementation is correct I can see this as a good plugin for
> CouchDB. Once it's written all you need is a bunch of people to use it
> and like it, and it could be a solid candidate for inclusion in core.
>
> This thread dearly belongs on dev@ - cross posting this message, let's
> try to keep subsequent replies on dev@.
>
> Cheers,
> Chris
>
>
>   


Mime
View raw message