couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Henri van den Bulk <hvandenb...@gmail.com>
Subject Re: Reconciling Data
Date Thu, 03 Nov 2011 14:21:50 GMT
Thanks for the tip - we'll give it a shot.

On Nov 2, 2011, at 4:29 PM, Jens Alfke wrote:

> 
> On Nov 2, 2011, at 2:07 PM, Henri van den Bulk wrote:
> 
> Unfortunately the source system does not keep track of changes so there is no way of
knowing what the changes were. We've toyed around to maybe putting the data in an intermediate
database from which we can then do sql queries like not in and hash compares. However, this
seems to defeat the purpose of having couch.
> 
> If you get the doc IDs from the source system, sort them and write them one per line
to a simple text file, you can then efficiently compare two versions of that file (by reading
them line by line in parallel) to find which IDs have been added or removed in the interim.
You could practically do this with a shell script (using the sort and diff tools).
> 
> —Jens


Mime
View raw message