Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of kxepal@gmail.com designates
 74.125.82.169 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAD2UGCVxyVLw2vR97q-N3kVPku4J9OzTQcxpeZ0XMbd+UocEOg@mail.gmail.com>
References: 
 <CAHdjip+LFy82jJ51+0PuVp8iVv=Lm0j_mCi8BCucWhkObn5cqA@mail.gmail.com>
	<77579B29-B4E6-4F06-A3D5-A7D7B360BD02@couchbase.com>
	<CAHdjip+0GqbDDfKgYKjKJWp6uO6RBa+aNBgTFMcDoGhQAMGnZg@mail.gmail.com>
	<CAJNb-9o0wEmBmWvJw1jwOBJzr0_LUYUhmc52mcasdu=piSid2w@mail.gmail.com>
	<CAHdjipJYF0quPs5rR6fMjw8mvs2NtLQXG1wojSpVFczQz2Zr7A@mail.gmail.com>
	<CAD2UGCVxyVLw2vR97q-N3kVPku4J9OzTQcxpeZ0XMbd+UocEOg@mail.gmail.com>
Date: Thu, 21 Nov 2013 11:34:53 +0400
Message-ID: 
 <CAHdjipLHyV5_Wxg1s6-U8LB=48bQdCp+CXOOWHdZ4M+fDuiD4g@mail.gmail.com>
Subject: Re: [REVIEW] CouchDB Replication Protocol
From: Alexander Shorin <kxepal@gmail.com>
To: "dev@couchdb.apache.org" <dev@couchdb.apache.org>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Hi Dale,

Thanks for review!

I agree, that there is need high level overview and reduce details.
I'll take pause for few weeks to found good idea how to reorganise
this document. For instance, API reference should be on top of the
document while there shouldn't be too much details about
couch_replicator routines.

But indeed, this is how CouchDB replication works. This means, that by
implementing the same API your peer would be able to handle push
replication and run pull with easy. Otherwise, you're going to break
the protocol.

For instance, you say PouchDB doesn't work multipart/mixed content.
That's might be ok for PouchDB-to-CouchDB replication, but if you run
CouchDB-to-PouchDB it will fail due to lack of such support. By the
same reason it will fail if you don't implement _ensure_full_commit
endpoint.

So, let's say there are two cases: when replication is served by
CouchDB and when it's served by some other app (PouchDB, CouchBase
Lite, etc.) For the last case you both are right: there is no need to
implement all these things to have replication works and you may use
any other CouchDB API to sync the documents. But for the first to have
complete support pull and push replication you should do this
otherwise replication will fall.

I understand now all problems of my document (there is also lack of
BigCouch/Cloudant specific), but where I should draw the line? During
implementation of replipy (test dummy peer in python that is able to
sync with CouchDB, but not host replication itself) I'd hit all these
caveats, details and specifics - that's why I drop both DataProtocol
and TouchDB algorithms. They are simple, but useless to make something
from scratch. So from point of CouchDB you must implement all required
API endpoints and content-type handling. But from point of PouchDB,
CouchBase Lite, Cloudant etc. it seems that you may have different
endpoints and replication will works different from couch_replicator.

So I'm really confused how replication protocol should looks, but I
feel that it must completely describe what happens when pull and push
replication process is served by CouchDB, not by anything else.

But I would be love to see any other ideas since this is just my point
of view on this problem (:

Thanks!


--
,,,^..^,,,


On Wed, Nov 20, 2013 at 5:44 PM, Dale Harvey <dale@arandomurl.com> wrote:
> Sorry for taking a while to look at this, busy with couchdb conf etc
>
> In general I agree with Jens, this serves as great documentation for how
> CouchDB's replication works, but its very detailed without a high level
> overview
>
> The PouchDB replicator was done following what Jens documented @
> https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replication-Algorithmwh=
ich
> was invaluable
>
> Another problem is that its difficult to distinguish between an
> implementation and a spec, a session id isnt needed for a pull replicatio=
n,
> revs diffs arent always needed, the details around the HTTP semantics
> (especially the multipart/mixed) stuff is not relevant to PouchDB as Jens
> mentioned, things like ensure_full_commit are quite specific
>
> I think one major improvement would be to restore the original high level
> documentation (as seen @
> https://github.com/couchbaselabs/TouchDB-iOS/wiki/Replication-Algorithm o=
r
> http://dataprotocols.org/couchdb-replication/) and use the current detail=
ed
> sections as additional references
>
> As it is, one problem I can see would be: 4.2.4.1. Verify Peers would
> suggest that its impossible to do pull replication if you dont have write
> permissions on the database, I am not sure if thats because I am havig a
> hard time following the diagram or thats what it is actually saying
>
> Thanks a lot for this work
>
>
>
>
>
> On 12 November 2013 06:21, Alexander Shorin <kxepal@gmail.com> wrote:
>
>> On Tue, Nov 12, 2013 at 9:35 AM, Benoit Chesneau <bchesneau@gmail.com>
>> wrote:
>> > On Tuesday, November 12, 2013, Alexander Shorin wrote:
>> >> https://github.com/kxepal/couchdb-replication-protocol
>> >
>> > hrmmm why another repo? i have created the hacking repo on my github f=
or
>> > that... Or we can like we said put all in a couchdb branch and
>> collaborate
>> > via git/github. At least i thought we decided that...
>>
>> Errr...that was unwise step from my side. Moved to CouchDB feature
>> branch. Sorry!
>>
>> > will review in another mail.
>> >
>> > - beno=C3=AEt
>>
>> Thanks!
>>
>> --
>> ,,,^..^,,,
>>