From user-return-5839-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Thu Aug 06 23:22:47 2009 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 39394 invoked from network); 6 Aug 2009 23:22:47 -0000 Received: from hermes.apache.org (HELO mail.apache.org) (140.211.11.3) by minotaur.apache.org with SMTP; 6 Aug 2009 23:22:47 -0000 Received: (qmail 81915 invoked by uid 500); 6 Aug 2009 23:22:53 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 81834 invoked by uid 500); 6 Aug 2009 23:22:53 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 81824 invoked by uid 99); 6 Aug 2009 23:22:53 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Aug 2009 23:22:53 +0000 X-ASF-Spam-Status: No, hits=-1.0 required=10.0 tests=RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: local policy) Received: from [66.111.4.28] (HELO out4.smtp.messagingengine.com) (66.111.4.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 06 Aug 2009 23:22:41 +0000 Received: from compute2.internal (compute2.internal [10.202.2.42]) by gateway1.messagingengine.com (Postfix) with ESMTP id 26FD3E944; Thu, 6 Aug 2009 19:22:20 -0400 (EDT) Received: from heartbeat2.messagingengine.com ([10.202.2.161]) by compute2.internal (MEProxy); Thu, 06 Aug 2009 19:22:20 -0400 X-Sasl-enc: 2Vaind5rwEaQT4PSikysuZY4RjvZY3KENB8LnWmqdTUU 1249600939 Received: from NitinBorwankarsComputer.local (c-71-202-180-50.hsd1.ca.comcast.net [71.202.180.50]) by mail.messagingengine.com (Postfix) with ESMTPSA id 2F9513AD3 for ; Thu, 6 Aug 2009 19:22:19 -0400 (EDT) Message-ID: <4A7B65AA.3030005@borwankar.com> Date: Thu, 06 Aug 2009 16:22:18 -0700 From: Nitin Borwankar User-Agent: Thunderbird 2.0.0.22 (Macintosh/20090605) MIME-Version: 1.0 To: user@couchdb.apache.org Subject: Re: MIME dump/load and implications References: <921000908061346p38fe9d4arf292d7fae14dafa0@mail.gmail.com> <4A7B4AA1.4090707@borwankar.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Chris Anderson wrote: > On Thu, Aug 6, 2009 at 2:41 PM, Adam Kocoloski wrote: > >> Hi Nitin, sure. I think Paul just meant that if we wrote it for >> replication, anyone could use the same facility to build something like what >> you're talking about. After all, the replicator just speaks to CouchDB >> servers using the same HTTP requests as everyone else. >> >> I'm sure jchris knows better whether mimeparse would be suitable for this. >> > > mimeparse.js is just a library for handing Accept headers. It's got > nothing to do with the multipart format. > I just saw that in the meanwhile so yeah that's not useful. > on the rest of your topic, does storing emails as attachments do the trick? > > not quite - a MIME message has it's own attachments - a MIME message = a doc not an attachment. > also, I'm of the opinion that the rest of the world should learn to > speak JSON. :) > Yes and please don't misunderstand the email to mean MIME is better/worse/a substitute for JSON. I am working very hard to move the entire world of academic bibliographic data in bibtex to JSON, so I come here not to bury JSON but to praise him :-) > more pragmatically, if you wanted to act as a smtp server, you'd > probably be able to wrap one around couch pretty easily. I've seen it > done in erlang (sorry, closed-source) but I'm guessing that > integrating Lamson wouldn't be hard. > > Again - never mentioned SMTP. Moving MIME documents as data between endpoints can be done independent of SMTP, hopefully over REST HTTP transfer. I can take a floppy and put a bunch of MIME docs with the right headers in a file with an inbox format, fly to the other end of the world in an airplane - deliver it and someone can open it up in Thunderbird with no knowledge of the airplane-floppy protocol. Data and Protocol are orthogonal in email. In fact before the days of DNS and SMTP people did email with UUCP. Painful but it worked. Data and protocol are orthogonal in email - we tend to confuse and conflate them all the time. > http://lamsonproject.org/ > > Yeah Zed is doing amazing things with Python SMTP - but not referring in SMTP in this discussion - just in new ways to move MIME docs around *without* SMTP :-). Want to act not as an SMTP server but as a web-MIME-interchange platform. A platform that maybe has access to an external web-SMTP gateway somewhere else so it can inject email into the existing mail infrastructure. But it can also do p2p messaging over HTTP independent of SMTP, between trusted auth'ed collections of endpoints. Hopefully someday the SMTP gateway may be less useful. Nitin > Chris > > > >> Cheers, >> >> Adam >> >> On Aug 6, 2009, at 5:26 PM, Nitin Borwankar wrote: >> >> >>> Hi Paul, >>> >>> I never used the word replication - it should be possible to create a REST >>> based couchapp driven MIME transfer p2p web quiteindependent of replication >>> which is also cool. Front the couchapp with the usual auth-proxy stuff for >>> now so only auth'ed people can communicate with you. >>> >>> Just replace JSON with MIME in all the reference docs and make the URL's >>> point to a design doc that does the transformations. >>> >>> On the way out it could be just _shows or an _list that takes multiple >>> objects and wraps them as a mime multipart. >>> >>> On the way in set up some REST endpoints that take POST's, parse mime >>> multiparts ( jchris's mimeparser?) convert to Couch docs, manage >>> attachments and puts them in _attachments ... and we're off to the races - >>> free user controlled, MIME-and-Mail-as-aplatform driven webmail apps for >>> all. >>> Yay Couch! >>> >>> Nitin >>> >>> >>> Paul Davis wrote: >>> >>>> Definitely some interesting points here. There have been discussions >>>> on using multipart-mime messaging in the replication protocol which >>>> could setup for some interesting prospects like this. I'm not sure on >>>> specifics in terms of replication, but having an endpoint that allows >>>> edits via multipart-mime could be a very fun thing to play with. >>>> >>>> Also, AFAIK there's nothing that prevents an isomorphic >>>> representation. As you point out, couchdb-python handles everything >>>> just fine here. >>>> >>>> Paul Davis >>>> >>>> On Thu, Aug 6, 2009 at 4:46 PM, Nitin Borwankar >>>> wrote: >>>> >>>> >>>>> Hi guys, >>>>> >>>>> I see that the python based dump/load uses MIME multipart docs as an >>>>> on-disk >>>>> serialisation format for couchdb databases. >>>>> An overall question then arises - can CouchDB be considered a MIME >>>>> database >>>>> which oh also happens to talk JSON? >>>>> So before that - is there a 1-1 strong correspondence between a CouchDB >>>>> document and a MIME multipart, or are there things around the edges that >>>>> are >>>>> crufty - I would assume a strong correspondence since dump/load uses it >>>>> and >>>>> I haven't seen any caveast about document content that is not dumpable. >>>>> >>>>> So assuming the 1-1 correspondence - could one use some "translation >>>>> layer" >>>>> couchapp that accepts arbitrary content/type + multipart-mixed MIME >>>>> object >>>>> over HTTP and then transparenty serialise them to JSON underneath. >>>>> >>>>> Given that dump/load already does this - it would see that there are no >>>>> obvious glaring flaws in this logic - but I have been known to be wrong, >>>>> once :-). >>>>> >>>>> If this is indeed feasible - then each CouchDB + MIME-trans becomes a >>>>> web >>>>> mail node - and Couch begins to be the platform for a messaging >>>>> revolution >>>>> as well as an application revolution. I am thinking now not as CouchDB >>>>> for >>>>> backing up your email - but CouchDB as your mail client/server for p2p >>>>> MIME >>>>> based "email". >>>>> >>>>> Permissions etc are important to avoid complete disaster of course - but >>>>> private high quality communication that just reuses existing message >>>>> formats, with better storage and transport would seem like an idea whose >>>>> time has come a long time ago and has been knocking at the door for a >>>>> decade. >>>>> >>>>> Yes, yes, there's the issue of spam - so see the P.S. >>>>> >>>>> Just a few idle thoughts, >>>>> >>>>> Nitin >>>>> >>>>> P.S. Back in 1998 I tried to convince Sybase to have MIME as a native >>>>> type >>>>> in the db and it even got speced out ( I have the spec with the date on >>>>> it! >>>>> ) but got canned becous ethe VP of enginnering wanted to know "what was >>>>> the >>>>> market exactly for this kind of stuff". Other than that I was granted >>>>> a >>>>> patent for doing p2p discussions over email back in 2003 - I let it >>>>> expire >>>>> for multiple reasons. So I am somewhat non-naive about and aware of the >>>>> issues and pitfalls around this sort of thinking. At the same time I am >>>>> of >>>>> the strong belief that when one looks at messages as data to be moved >>>>> around >>>>> between endpoints with well defined addressing schemes, and one ignores >>>>> the >>>>> protocols for a bit, then all sorts of fun things start to happen. >>>>> >>>>> >>>>> 37% of all statistics are made up on the spot >>>>> >>>>> ------------------------------------------------------------------------------------- >>>>> Nitin Borwankar >>>>> nborwankar@gmail.com >>>>> >>>>> >>>>> >> > > > >