Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: local policy)
Message-ID: <4A7B6CA5.9060904@borwankar.com>
Date: Thu, 06 Aug 2009 16:52:05 -0700
From: Nitin Borwankar <nitin@borwankar.com>
User-Agent: Thunderbird 2.0.0.22 (Macintosh/20090605)
MIME-Version: 1.0
To: user@couchdb.apache.org
Subject: Re: MIME dump/load and implications
References: <921000908061346p38fe9d4arf292d7fae14dafa0@mail.gmail.com>
	 <e2111bbb0908061352h33437919ic5d88b05b76d1e25@mail.gmail.com>
	 <4A7B4AA1.4090707@borwankar.com>
	 <E84EA6CF-313E-40A7-B907-1C73385196BD@apache.org>
 <e282921e0908061537q2232409ay61003db280324712@mail.gmail.com>
 <4A7B65AA.3030005@borwankar.com>
In-Reply-To: <4A7B65AA.3030005@borwankar.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit

Nitin Borwankar wrote:


Coincidentally/serendipitiously this turned up in my twitter stream - 
http://code.google.com/p/webfinger/ .

Nitin


> Chris Anderson wrote:
>> On Thu, Aug 6, 2009 at 2:41 PM, Adam Kocoloski<kocolosk@apache.org> 
>> wrote:
>>  
>>> Hi Nitin, sure.  I think Paul just meant that if we wrote it for
>>> replication, anyone could use the same facility to build something 
>>> like what
>>> you're talking about.  After all, the replicator just speaks to CouchDB
>>> servers using the same HTTP requests as everyone else.
>>>
>>> I'm sure jchris knows better whether mimeparse would be suitable for 
>>> this.
>>>     
>>
>> mimeparse.js is just a library for handing Accept headers. It's got
>> nothing to do with the multipart format.
>>   
>
> I just saw that in the meanwhile so yeah that's not useful.
>
>> on the rest of your topic, does storing emails as attachments do the 
>> trick?
>>
>>   
>
> not quite - a MIME message has it's own attachments - a MIME message = 
> a doc not an attachment.
>> also, I'm of the opinion that the rest of the world should learn to
>> speak JSON. :)
>>   
> Yes and please don't misunderstand the email to mean MIME is 
> better/worse/a substitute for JSON.
> I am working very hard to move the entire world of academic 
> bibliographic data in bibtex to JSON, so I  come here not to bury JSON 
> but to praise him :-)
>> more pragmatically, if you wanted to act as a smtp server, you'd
>> probably be able to wrap one around couch pretty easily. I've seen it
>> done in erlang (sorry, closed-source) but I'm guessing that
>> integrating Lamson wouldn't be hard.
>>
>>   
> Again - never mentioned SMTP.  Moving MIME documents as data between 
> endpoints can be done independent of SMTP, hopefully over  REST HTTP 
> transfer.
>
> I can take a floppy and put a bunch of MIME docs with the right 
> headers in a file with an inbox format, fly to the other end of the 
> world in an airplane - deliver it and someone can open it up in 
> Thunderbird with no knowledge of the airplane-floppy protocol.  Data 
> and Protocol are orthogonal in email.
> In fact before the days of DNS and SMTP people did email with UUCP.  
> Painful but it worked.
> Data and protocol are orthogonal in email - we  tend to confuse and 
> conflate them all the time.
>
>> http://lamsonproject.org/
>>
>>   
> Yeah Zed is doing amazing things with Python SMTP - but not referring 
> in SMTP in this discussion - just in new ways to move MIME docs around 
> *without* SMTP :-).
>
> Want to act not as an SMTP server but as a web-MIME-interchange 
> platform. A platform that maybe has access to an external web-SMTP 
> gateway somewhere else so it can inject email into the existing mail 
> infrastructure. But it can also do p2p messaging over HTTP independent 
> of SMTP,  between trusted auth'ed collections of endpoints.
>
> Hopefully someday the SMTP gateway may be less useful.
>
> Nitin
>
>> Chris
>>
>>
>>  
>>>  Cheers,
>>>
>>> Adam
>>>
>>> On Aug 6, 2009, at 5:26 PM, Nitin Borwankar wrote:
>>>
>>>    
>>>> Hi Paul,
>>>>
>>>> I never used the word replication - it should be possible to create 
>>>> a REST
>>>> based couchapp driven MIME transfer p2p web quiteindependent of 
>>>> replication
>>>> which is also cool.  Front the couchapp with the usual auth-proxy 
>>>> stuff for
>>>> now so only auth'ed people can communicate with you.
>>>>
>>>> Just replace JSON with MIME in all the reference docs and make the 
>>>> URL's
>>>> point to a design doc that does the transformations.
>>>>
>>>> On the way out it could be just _shows or an _list that takes multiple
>>>> objects and wraps them as a mime multipart.
>>>>
>>>> On the way in set up some REST endpoints that take POST's, parse mime
>>>> multiparts ( jchris's mimeparser?)  convert to Couch docs, manage
>>>> attachments and puts them in _attachments ... and we're off to the 
>>>> races -
>>>> free user controlled, MIME-and-Mail-as-aplatform driven webmail 
>>>> apps for
>>>> all.
>>>> Yay Couch!
>>>>
>>>> Nitin
>>>>
>>>>
>>>> Paul Davis wrote:
>>>>      
>>>>> Definitely some interesting points here. There have been discussions
>>>>> on using multipart-mime messaging in the replication protocol which
>>>>> could setup for some interesting prospects like this. I'm not sure on
>>>>> specifics in terms of replication, but having an endpoint that allows
>>>>> edits via multipart-mime could be a very fun thing to play with.
>>>>>
>>>>> Also, AFAIK there's nothing that prevents an isomorphic
>>>>> representation. As you point out, couchdb-python handles everything
>>>>> just fine here.
>>>>>
>>>>> Paul Davis
>>>>>
>>>>> On Thu, Aug 6, 2009 at 4:46 PM, Nitin Borwankar<nitin@borwankar.com>
>>>>> wrote:
>>>>>
>>>>>        
>>>>>> Hi guys,
>>>>>>
>>>>>> I see that the python based dump/load uses MIME multipart docs as an
>>>>>> on-disk
>>>>>> serialisation format for couchdb databases.
>>>>>> An overall question then arises - can CouchDB be considered a MIME
>>>>>> database
>>>>>> which oh also happens to talk JSON?
>>>>>> So before that - is there a 1-1 strong correspondence between a 
>>>>>> CouchDB
>>>>>> document and a MIME multipart, or are there things around the 
>>>>>> edges that
>>>>>> are
>>>>>> crufty - I would assume a strong correspondence since dump/load 
>>>>>> uses it
>>>>>> and
>>>>>> I haven't seen any caveast about document content that is not 
>>>>>> dumpable.
>>>>>>
>>>>>> So assuming the 1-1 correspondence - could one use some "translation
>>>>>> layer"
>>>>>> couchapp that accepts arbitrary content/type + multipart-mixed MIME
>>>>>> object
>>>>>> over HTTP and then transparenty serialise them to JSON underneath.
>>>>>>
>>>>>> Given that dump/load already does this - it would see that there 
>>>>>> are no
>>>>>> obvious glaring flaws in this logic - but I have been known to be 
>>>>>> wrong,
>>>>>> once :-).
>>>>>>
>>>>>> If this is indeed feasible - then each CouchDB + MIME-trans 
>>>>>> becomes a
>>>>>> web
>>>>>> mail node - and Couch begins to be the platform for  a messaging
>>>>>> revolution
>>>>>> as well as an application revolution. I am thinking now not as 
>>>>>> CouchDB
>>>>>> for
>>>>>> backing up your email - but CouchDB as your mail client/server 
>>>>>> for p2p
>>>>>> MIME
>>>>>> based "email".
>>>>>>
>>>>>> Permissions etc are important to avoid complete disaster of 
>>>>>> course - but
>>>>>> private high quality communication that just reuses existing message
>>>>>> formats, with better storage and transport would seem like an 
>>>>>> idea whose
>>>>>> time has come a long time ago and has been knocking at the door 
>>>>>> for a
>>>>>> decade.
>>>>>>
>>>>>> Yes, yes, there's the issue of spam - so see the P.S.
>>>>>>
>>>>>> Just a few idle thoughts,
>>>>>>
>>>>>> Nitin
>>>>>>
>>>>>> P.S.  Back in 1998 I tried to convince Sybase to have MIME as a 
>>>>>> native
>>>>>> type
>>>>>> in the db and it even got speced out ( I have the spec with the 
>>>>>> date on
>>>>>> it!
>>>>>> ) but got canned becous ethe VP of enginnering wanted to know 
>>>>>> "what was
>>>>>> the
>>>>>> market exactly for this kind of stuff".  Other than that I was 
>>>>>> granted
>>>>>>  a
>>>>>> patent for doing p2p discussions over email back in 2003 - I let it
>>>>>> expire
>>>>>> for multiple reasons.  So I am somewhat non-naive about and aware 
>>>>>> of the
>>>>>> issues and pitfalls around this sort of thinking. At the same 
>>>>>> time I am
>>>>>> of
>>>>>> the strong belief that when one looks at messages as data to be 
>>>>>> moved
>>>>>> around
>>>>>> between endpoints with well defined addressing schemes, and one 
>>>>>> ignores
>>>>>> the
>>>>>> protocols for a bit, then all sorts of fun things start to happen.
>>>>>>
>>>>>>
>>>>>> 37% of all statistics are made up on the spot
>>>>>>
>>>>>> ------------------------------------------------------------------------------------- 
>>>>>>
>>>>>> Nitin Borwankar
>>>>>> nborwankar@gmail.com
>>>>>>
>>>>>>
>>>>>>           
>>>     
>>
>>
>>
>>   
>