Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of ryan.ramage@gmail.com
 designates 209.85.212.52 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <7F5F85A9-D5B6-4779-A523-ECD6E90CBDD0@couchbase.com>
References: <B0E987C1-51B1-4ABD-97C2-2CF4198E1B4F@gmail.com>
	<7F5F85A9-D5B6-4779-A523-ECD6E90CBDD0@couchbase.com>
Date: Wed, 2 Nov 2011 14:35:08 -0600
Message-ID: 
 <CAL-h1aQ16zW_FJugARvJy2tLqnkhYvhOv3Q+8qfpMiDDR2LMXw@mail.gmail.com>
Subject: Re: Reconciling Data
From: Ryan Ramage <ryan.ramage@gmail.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: quoted-printable

You might want to have your other data source more sleep-y (no joke)

http://syncable.org/


On Wed, Nov 2, 2011 at 1:49 PM, Jens Alfke <jens@couchbase.com> wrote:
>
> On Nov 2, 2011, at 9:20 AM, Henri van den Bulk wrote:
>
>> I'm in the process of writing an external program in Java that reconcile=
s data in CouchDB to a source system. One of the basic parts is to determin=
e what data needs to be removed from CouchDB. The good thing is that the Id=
s in CouchDB are the same as the Ids in the source system. However, some in=
itial test seem that the process is very slow in determining what needs to =
be removed.
>
> It=92s inevitably going to be inefficient to have to fetch _all_ the doc =
IDs from CouchDB and _all_ the IDs from the other system and then scan them=
 looking for differences.
>
> Instead, is there a way you can query the other system to find out which =
docs have been removed since the last time you synced?
>
> =97Jens


--=20
Twitter: @eckoit