couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wendall Cada <wenda...@apache.org>
Subject Re: Google Summer of Code topics
Date Sat, 23 Mar 2013 00:07:10 GMT
On 03/22/2013 12:48 PM, Dave Cottlehuber wrote:
> On 22 March 2013 19:14, Mark Hahn <mark@hahnca.com> wrote:
>>>   Implement partial reads and updates of documents,
>> In case anyone didn't know, you can do partial updates right now with an
>> update handler.  I have been using one for some time that allows the app to
>> modify any part of a doc with a single http request.  It even allows one to
>> modify an attribute nested inside objects.  I've ended up using only this
>> for all updates.
> Hi Mark,
>
> Completely true. Maybe you would like to contribute that back perhaps?
> Even as a wiki entry that would be useful for folk, and I'm happy to
> help clean up / document if required.
>
> I was thinking at the time of the JSON PATCH draft RFC, aka
>
> http://www.mnot.net/blog/2012/09/05/patch
> http://tools.ietf.org/html/rfc5789
> http://tools.ietf.org/html/draft-ietf-appsawg-json-patch-10
> https://github.com/json-patch/json-patch-tests
> https://npmjs.org/package/json-patch
>
> Did you use this approach, or tackle it differently?
>
> Another idea that I had on the back burner was implementing
> cryptographic support for doc signing, e.g. pgp-like signing and
> validation, or even encryption. I think Jim Klo has done this or
> something similar already.
>
> A+
> Dave
I think this idea is really a great one. Every time I bring this up, the 
response is "Why don't you just send the entire doc?". We'll, I could 
give a zillion reasons at least, but here are the main ones:

1. I want to be able to limit what I send over the wire without 
resorting to the latest fad in over the wire json compression 
techniques. Just simply being able to send my actual changes will beat 
every one of these methods 100% of the time.
2. I want to only share a subset of the actual doc with the client-side app.
3. Have the ability to store a history of doc changes easily.

Imagine a scenario where all changes are stored as attachments and I can 
easily replay events over the lifetime of a doc, or a truncated history, 
depending on how long you want to store this for. Full replicatable 
(sp?) history for an entire db!

I've implemented quite a few solutions in some test code over the last 
several months. What I've found is that diff is the slowest possible way 
to do this. Maybe some of these solutions are faster now, but was crazy 
slow when I looked at it about six months ago.

I saw some very clever OT (Operation Transformation) techniques in use. 
There is currently a really neat github project that has been 
maintaining some very good implementations. Of particular interest is 
the javascript implementation 
http://operational-transformation.github.com/ot-for-javascript.html

On that same site, there is quite a bit of information about OT. I 
really think it would be awesome for a SoC project to have this 
implemented in Erlang. There is some long-term potential of possibly 
speeding up replication utilizing OT/diffs.

Wendall



Mime
View raw message