Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of dionne@dionne-associates.com
 designates 69.89.21.17 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=default;
 d=dionne-associates.com;
	h=Received:Mime-Version:In-Reply-To:References:Content-Type:Message-Id:Content-Transfer-Encoding:From:Subject:Date:To:X-Mailer:X-Identified-User;
	b=jQCGmJ79jA1aJutyYXUl/P0S/wHj7tWN1RYzQgD13a1jWB9AvQAsRQSX0JjRthNf4pP8kRXXXYlGCb6tUlzUxgfS93hFC5PRA6OkK+hsaY1AbgEPEcicV9+M/ld1iowt;
Mime-Version: 1.0 (Apple Message framework v753.1)
In-Reply-To: <49859AB5.7080404@diskware.net>
References: <498464EC.1070209@diskware.net>
	 <e2111bbb0901311956x5040956elf76c3fe17e7526bd@mail.gmail.com>
 <e282921e0901312030o30f6abawb4e6905fcb048bc4@mail.gmail.com>
 <49859AB5.7080404@diskware.net>
Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
Message-Id: <5FCD4937-E8BD-4024-9E46-A24061F4D322@dionne-associates.com>
Content-Transfer-Encoding: 7bit
From: Robert Dionne <dionne@dionne-associates.com>
Subject: Re: couch_gen_btree: pluggable storage / tree engines
Date: Sun, 1 Feb 2009 08:21:53 -0500
To: dev@couchdb.apache.org

Martin,

   I'm very keen on relationships between documents. Coming from the  
description logic community, I'd like to allow users to declare  
certain fields that relate documents and then compute transitive  
closures over dags whose nodes are documents and whose arcs the  
fields of interest. This goes against the grain of  couchdb as  
collections of unrelated documents, I know, but it's what I want to  
do as couchdb's schema-less design offers many advantages over  
relational databases. Relational databases aren't that great for  
storing graphs either.

   I don't need to run full classification algorithms in the document  
store, but would like to just maintain relationships (user-defined)  
and transitive closures of them. Inferencing would perhaps be better  
done externally similar to the hypercouch work. So this would best be  
served by pluggable indexing and maybe pluggable storage, though I  
think I could live without the latter for now.

   So I'm very excited about your ideas. I too have been reviewing  
the code with this in mind and I would agree with others that it's  
perhaps a post 1.0 task. From the little time I've spent chasing down  
a couple of bugs I've seen there are a few subtle aspects to it. I've  
also noticed that the style of design in this community is more  
bottoms up, which is how it should be when building something new, so  
prototypes are perhaps better for fleshing out ideas. Anyway I'm very  
happy to help an d collaborate on this as I can.

Cheers,

Bob

Robert Dionne
Chief Programmer
dionne@dionne-associates.com
203.231.9961


On Feb 1, 2009, at 7:51 AM, Martin Scholl wrote:

> Chris Anderson wrote:
>> On Sat, Jan 31, 2009 at 7:56 PM, Paul Davis  
>> <paul.joseph.davis@gmail.com> wrote:
>>> Martin,
>>>
>>> Very cool ideas. We've been discussing erlang plugins. The
>>> conversation has generally gotten as far as, "erlang plugins... yeah
>>> we should have those."
>>
>> I agree this is cool, but I think it would be healthier for the
>> project to wait until we release a rock-solid 1.0.
>>
>> There are some incredibly non-obvious things happening inside, and a
>> big disruption right now wouldn't necessarily keep them all in
>> balance. Once we've met 1.0, we'll have a solid basis for comparison,
>> of any alternate implementations.
>>
>> Then, let the fun begin. :)
>>
>> Martin, I'd very much like to hear more about the sorts of indexers
>> you'd build. Sounds exciting.
> I'd like to experiment with Merkle trees, because these could turn out
> to be a good foundation for several use-cases:
> - index/tree-synchronization: replication is trivial with merkle  
> trees,
> only changed parts of the tree get replicated in a secure manner.
> - secure document storage: modified documents (disc corruption, sw
> failure or even the "bad cracker"-case)
> - by using GPG/PGP-signatures probably even cryptographical secure
> design doc code signing, e.g. "safe applications"
>
> Furthermore, there are a lot of other clever map data structures
> available (not in the sense of a->b , but a<->b) which could become
> quite handy to store document relationships. I'm sure, the database  
> ppl
> out here have many more ideas about what could be added to CouchDB.
>
>
> Martin