Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of randall.leeds@gmail.com
 designates 209.85.210.180 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <A5640261771E4D8BB679906791E611DD@zive.at>
References: <D6831D11FC974DD9AB519572990939F5@zive.at>
	<CAONf+LSE7-39kw3_ZjV3EexCZBiN6x1rVO5ysn=N5uUpQqv04Q@mail.gmail.com>
	<A5640261771E4D8BB679906791E611DD@zive.at>
Date: Mon, 1 Aug 2011 15:29:39 -0700
Message-ID: 
 <CAAL6JQjax1UOVw+M=PMNOSvkCRno+Gnf=AGE+42Zf8Gi1=VCHw@mail.gmail.com>
Subject: Re: Guys with deep knowledge about CouchDB Replication, I'd need your
 help :)
From: Randall Leeds <randall.leeds@gmail.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Fri, Jul 29, 2011 at 04:12, Michael Aufreiter <ma@zive.at> wrote:
>
> Nice. Having a look at it.
>
> It really depends on how much functionality you wanna delegate to the cli=
ent. In my opinion, in most cases you wan't to keep the amount of local dat=
a low. That's why I'll probably use localstorage to memoize a complete snap=
shot of the current graph. Once you reload the page all data is loaded into=
 memory again (restored) and you can query it as usual (using Data.Graph#fi=
nd). So in my case I'd rather wanna use just LocalStorage without employing=
 indexedDB etc. as local views(map-reduce etc) wouldn't be an requirement h=
ere. But what I need to solve is the replication thing.

To solve replication and safely allow the application to be used
across tabs you need indexedDB because it has transactions.

Let me break that deduction down:
Incremental replication requires a restart-able changes feed. A
changes feed requires an index of updates order by time/sequence
number. A sequence index that's kept in sync with the index of
documents ordered by name requires changes to two indexes in an atomic
transaction or race conditions are possible.

If you're not worried about replicating incrementally or you're not
worried about the possibility of race conditions (e.g., concurrent
access by multiple tabs) you might be able to use LocalStorage,
otherwise you'll need IndexedDB.

Since PouchDB aims to be a robust and general-purpose implementation
of CouchDB it uses IndexedDB.

-Randall

>
> On Thursday, July 28, 2011 at 6:18 PM, Max Ogden wrote:
>
> > beginnings of an html5 couch: https://github.com/mikeal/pouchdb
> >
> > it would be great to get @mikeal and @tilgovi to chime in on this
> > thread as they were writing the replicator for that
> >
> > On Thu, Jul 28, 2011 at 11:48 AM, Michael Aufreiter <ma@zive.at (mailto=
:ma@zive.at)> wrote:
> > > I'm currently working on a complete data-persistence solution for off=
line apps, involving CouchDB and Data.js. I already introduced Data.js here=
 at this mailing list the other day, but here's a link again:
> > >
> > > http://substance.io/#michael/data-js
> > >
> > > I've setup a cleanroom example (tasks) that I want to test the new sy=
nc-functionality against.
> > >
> > > http://tasks.substance.io (don't miss the sync button in the upper ri=
ght corner)
> > > https://github.com/michael/data/blob/7729d41677e48bd5132119997dc0cff5=
3522bb55/examples/tasks/public/javascripts/views/app.js
> > >
> > > It's currently just one way. It just writes changes to the server but=
 does not pull in node-updates. Now this should change.
> > >
> > > The algorithm for a bi-directional sync I have in mind looks like so:
> > >
> > > 1. Pull: For all nodes I have in my local graph, check if there are u=
pdates (other users might have updated them), and if yes, pull them in
> > > If conflicts occur the client/user decides how to resolve it (choose =
a revision or merge it)
> > >
> > > 2. Push: Write all local dirty nodes to the server
> > >
> > > If that succeeded, the sync is complete. Usually if there's not much =
time between the pull and the push it's unlikely to run into conflicts when=
 doing the push.
> > >
> > > However I'm asking myself how CouchDB replication is implemented -- m=
aybe I can re-use some of the concepts.
> > >
> > > In order to perform the Pull, I thought about sending a list of ID's+=
revisions to the server. The server (resp. Couch) should then check if ther=
e are updates for any of them. If yes, those nodes should be fetched and de=
livered to the client. Given that number of ID/revision pairs, what would b=
e the best way to check for updates? Or do you have any other ideas on how =
to do the pull?
> > >
> > > An implication of this scenario is that application developers should=
 do their best to keep the local graph rather small (the bigger it gets the=
 more overhead you have when doing the push, also more memory is used). How=
ever this should suit a lot of scenarios (like in my case making possible o=
ffline editing of Substance.io (http://Substance.io) documents)
> > >
> > > Would be great if some of you could help me out a bit here. I think s=
uch a framework (Data.js+Couch) would be a great benefit for application de=
velopers who wan't to build offline apps. What do you think?
> > >
> > > Thanks!
> > >
> > > -- Michael
>
>