incubator-wave-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael MacFadden <>
Subject Re: A Very Wavey Plan (P2P!)
Date Thu, 20 Jun 2013 02:57:37 GMT

We will put a wiki page together.  We will also have the discussion here.
That said, I would like to select at least three people who sign up to
shepherd the discussion.  Everyone is welcome and the committee members
won't have any specific authority.  I am just looking for people to form a
working group and commit to making sure we can define and meet some


On 6/19/13 6:25 PM, "Upayavira" <> wrote:

>I'd encourage you to fire up a wiki page and start the discussion here.
>I suspect due to te nature of the topic, participants will quickly be
>self selecting. 
>On Wed, Jun 19, 2013, at 10:11 PM, Joseph Gentle wrote:
>> On Wed, Jun 19, 2013 at 1:26 PM, Michael MacFadden
>> <> wrote:
>> >>To start, I want to build a generic P2P OT container. This is a simple
>> >>wrapper that contains a set of OT documents and defines a network
>> >>protocol for keeping them in sync. The container needs to be able to
>> >>talk to another instance of itself running somewhere else and
>> >>syncronize documents between the two instances.
>> >>
>> >>Thats all I want this container to do - it should be as lightweight as
>> >>possible, so we can port it to lots of different languages and
>> >>environments. I want that code running in websites, in giant server
>> >>farms, in vim, and everywhere in between. It won't have any database
>> >>code, network code, users or a user interface (though it'll need APIs
>> >>to support all of that stuff). At its core it just does OT + protocol
>> >>work to syncronize documents.
>> >
>> > I strongly suggest we separate the OT Algorithms, the application
>> > protocol, and the network transport layer.
>> The network transport layer should definitely be separated out. I'd
>> also like to separate out the OT functions themselves (a la
>> share/ottypes).
>> I'm imagining coupling the concurrency control system and the
>> application level protocol. This coupling might not be needed - I
>> don't know enough yet to be able to tell. I think we should focus on
>> figuring out what OT system(s) to use first.
>> >>The OT itself I imagine building along the lines of Torben Weis's P2P
>> >>OT theory that he made in Lightwave:
>> >> . Briefly, every operation gets a
>> >>hash (like git). We add tombstones to wave's OT type and remove
>> >>invertability, so the transform function supports TP2. We also add a
>> >>prune function (inverse transform) which allows the history list to be
>> >>reordered (so you don't have to transform out on every site). The hard
>> >>part is figuring out which operations to sync, and which operations
>> >>need to be reordered. I'd like to go over the details with Michael
>> >>MacFadden and anyone else who's interested - there may well be a
>> >>better system that we should use instead. If there is, I'd like to
>> >>know about it now.
>> >
>> > I think this is another interesting area.  One thing to consider is
>>that I
>> > am not sure if the linear model that wave used is even the right
>> > If we start manipulating things like JSON Objects, Java Object, XML,
>> > nested documents, a hierarchical path mechanism may be best.
>> Yes. For ShareJS's JSON OT type, operations are defined by a path (a
>> list) and an operation at that path. I'm not convinced by this design
>> anymore - we should definitely discuss it.
>> > My other
>> > instinct here is, we should make sure that we base the OT on something
>> > that has been proven to be correct.  There has been some work to
>> > OT systems and prove that they are correct and solve the proper OT
>> > and support the required properties.
>> >
>> > Two other things that we need to consider.  Beyond TP1 and TP2, there
>> > also IP1, IP2, and IP3 that you need to think about if you are going
>> > support group undo, which in my opinion wave needs to support.  If you
>> > can't undo things in collaboration, then you have problems.
>> Using tombstones, I don't know how you can implement IP3. I've a mind
>> to abandon formal invertibility, and simply rely on undo stacks for
>> user level actions. But if we can find an architecture which meets our
>> needs and can support invertibility, then that would be even better.
>> > Basically, I am not confident that we know that wave's OT or what is
>> > lightwave is really what we want.  I am not saying that they are not,
>>I am
>> > just saying that I don't think we really have stated what we NEED
>>from the
>> > OT and then proved that a particular approach salsifies those needs.
>> > Simply having a demo of something that works in a toy environment is
>> > likely to have holes that we don't see until much, much later in the
>> > development cycle.
>> >
>> > My recommendation here would be for us to form a small committee to
>>do a
>> > literature review on the topic and to foster some technical
>> > The result should be something in the wiki that lays out a plan that
>> > community can comment on.
>> I completely agree. I don't know enough about what our other choices
>> are. You're much better read than I am and you're better connected to
>> the academic OT community. Would you be interested in organising /
>> chairing this committee?
>> >>I'm also thinking about full end-to-end encryption. Especially in the
>> >>wake of the PRISM stuff, I'd quite like to make something secure.
>> >>Snowden: "Encryption works. Properly implemented strong crypto systems
>> >>are one of the few things that you can rely on. Unfortunately,
>> >>endpoint security is so terrifically weak that NSA can frequently find
>> >>ways around it." --
>> >>leblower
>> >>.
>> > I like this idea.  There is a paper from France that talks about
>> > hashing, digital signing, etc that proves the authenticity of
>> > that might also be worth looking at.  I will try to dig up the
>> Great!
>> -Joseph

View raw message