From user-return-18715-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Mon Nov 7 03:07:01 2011 Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id B60817654 for ; Mon, 7 Nov 2011 03:07:01 +0000 (UTC) Received: (qmail 48848 invoked by uid 500); 7 Nov 2011 03:07:00 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 48817 invoked by uid 500); 7 Nov 2011 03:06:59 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 48809 invoked by uid 99); 7 Nov 2011 03:06:59 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Nov 2011 03:06:59 +0000 X-ASF-Spam-Status: No, hits=-0.7 required=5.0 tests=FREEMAIL_FROM,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of adam.kocoloski@gmail.com designates 209.85.161.180 as permitted sender) Received: from [209.85.161.180] (HELO mail-gx0-f180.google.com) (209.85.161.180) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Nov 2011 03:06:54 +0000 Received: by ggnv5 with SMTP id v5so178106ggn.11 for ; Sun, 06 Nov 2011 19:06:33 -0800 (PST) Received: by 10.236.197.9 with SMTP id s9mr32263281yhn.71.1320635193543; Sun, 06 Nov 2011 19:06:33 -0800 (PST) Received: from [192.168.1.8] (c-76-119-89-178.hsd1.ma.comcast.net. [76.119.89.178]) by mx.google.com with ESMTPS id j25sm26186121yhm.12.2011.11.06.19.06.31 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 06 Nov 2011 19:06:32 -0800 (PST) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Apple Message framework v1084) Subject: Re: replicate A => B, but only docs that exist in B From: Adam Kocoloski In-Reply-To: Date: Sun, 6 Nov 2011 22:06:29 -0500 Content-Transfer-Encoding: quoted-printable Message-Id: References: To: user@couchdb.apache.org X-Mailer: Apple Mail (2.1084) On Nov 6, 2011, at 8:19 PM, Dave Cottlehuber wrote: > On 6 November 2011 14:14, Gregor Martynus wrote: >> *SCENARIO*: >>=20 >> Each user has its own DB ("B") of documents which replicates = continuously >> to a center DB ("A") which contains all documents from all users. = That >> works perfectly fine so far. >>=20 >> What I'd like to add now are "shared documents", which can be updated = from >> others in the center DB ("A"). This changes should be replicated back = to >> the owners DB ("B"). >>=20 >> Hope that makes sense so far? >>=20 >>=20 >> *IDEA*: >>=20 >> What I'd love to have would be as simple way where I could start a >> continuous replication "A" =3D> "B" which only replicates the docs = that do >> already exist in "B". Is there something like this? >=20 > I think you mean, only replicate updated documents that already exist > in B. No I don't think this is possible today directly within the > replicator. But you can ask the replicator to pull or push a specific > set of ids only. Wouldn't it be possible to write a validator that rejects new document = creations from a particular user, then supply that user's credentials in = the target specification for the replicator? >=20 >> I though about making a filtered replication "A" =3D> "B" where the = filter >> checks if the document exists in "B", but as far as I understand it, = the >> filter only has access to the doc from "A", not from "B", right? >>=20 >>=20 >> *NOTE*: >>=20 >> I know I could use a filter that checks for some kind of `owner_id` = in the >> documents, but each document has child elements which I want to keep >> atomic, and I do not want to add an `owner_id` attribute to all of = these. >>=20 >=20 > What about: >=20 > #1 client-driven pull. > run an external process (perhaps from within couchdb itself) that > queries localhost/b/_all_docs, then page through these (if needed), > asking the replicator to retrieve just those specific docs. I assume > you can set up a continuous replication with specific doc keys just as > any normal replication request. >=20 > #2 listen on changes feed. > get a list of localhost/b/_all_docs, then pass this to your filtered > changes feed (maybe event driven with https://github.com/hookio/couch > for example) and when the required docs turn up, call the replicator > to retrieve it. >=20 > A+ > Dave