incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: MIME dump/load and implications
Date Fri, 07 Aug 2009 00:00:25 GMT
On Aug 6, 2009, at 7:06 PM, Nitin Borwankar wrote:

> Adam Kocoloski wrote:
>> Hi Nitin, sure.  I think Paul just meant that if we wrote it for  
>> replication, anyone could use the same facility to build something  
>> like what you're talking about.
>
> Hi Adam, Paul,
>
> Yes certainly agreed in theory but in practice I see two major  
> obstacles, one social and the other technical to this approach.
>
> a) The number of people who can work on the replication code is a  
> handful and this hugely raises the bar and puts all this in the  
> domain of "someday-maybe" - "it will need to get on the release  
> schedule" etc. which is less attractive to me.
>
> On the other hand doing it in userland via couchapp throws open the  
> gates for anyone to start experimenting with this and then the best  
> stuff will emerge and get traction, one hopes.  I couldn't possibly  
> make any useful contribution at the replication level at my current  
> level of knowledge of Couch/Erlang/... but hacking at the couchapp  
> level is not out of the range of my capabilities.  I suspect most  
> users of Couch are in the same boat as me.
>
> So I deliberately focus on a REST based MIME-payload protocol that  
> could run in parallel with the REST based JSON-payload protocol.
> At this level of abstraction it is easy to understand and talk about  
> for a much huger audience.  Moreover the amount of work needed to  
> bridge the gap between the already working python code and a  
> couchapp/javascript impl of this is much smaller.
>
>
> b)  This will all also be possible in the replication layer but a  
> messaging system has additional requirements such as keeping  
> cumulative track of where the message came from - adding a MIME- 
> header (JSON attribute)  i.e modifying the doc at each hop etc.   
> These concerns are not fundamental to point-point doc replication  
> and need to be layered on top/in addition to replication.   
> Replication needs to faithfully reproduce a doc from one endpoint to  
> another without the kinds of modification that happen to headers  
> (metadata attributes) in message transport.
>
> So thinking of messaging as "merely" replication glosses over some  
> fundamental issues that distinguish message documents in motion,  
> from more general documents.  Message documents have container level  
> attributes that get modified during the process of store-and-forward  
> to track the motion of the doc.
> Generic documents in CouchDB have no such requirement and it may be  
> counterproductive to intermingle the requirements.
>
> In fact it may be possible to make a stronger statement - Couch  
> replication should focus single mindedly on replication of documents  
> in a pt-to-pt fashion.
> Messaging semantics can then be layered on top of replication, in an  
> upper layer if desired - or not.  Injecting messaging concerns  
> (especially modification of container attributes) into replication  
> is a distraction of focus for replication, IMHO and possibly counter  
> to the requirements of faithful replication.
>
> Hope this clarifies why I reacted somewhat strongly to conflating  
> REST based messaging with replication - they overlap but should not  
> be equated if we want to build a messaging system, != a peer to peer  
> doc sharing system.

Hi Nitin, indeed it does clarify things.  I read too quickly the first  
time around and missed that key word -- couchapp.  I thought you were  
asking for Couch to natively speak MIME multipart as a prerequisite  
for the REST messaging system.  Best,

Adam

P.S. Do you have any BibJSON couchapps?  The .bib file for my  
dissertation is growing by the day.

Mime
View raw message