couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Barnes <>
Subject Managing synchronisation with an external data source
Date Mon, 23 Nov 2009 02:10:26 GMT
I'm building a couchdb DB to store and manage user, group and role data.
I have an external feed for the data.

The external data is delivered as a file containing all the records for 
each data type, daily.
I need to keep the information in the database current:
  - if there is a user defined in the current data that doesn't yet 
exist in the database, a document needs to be created.
  - if there a feed-created user in the database that doesn't exist in 
the current data, that user document needs to be set 'inactive'.

Similar logic needs to exist for the groups and roles. (Whether roles 
should be stored in the user document or separately, I'm not yet sure)

Given that there are ~200k users, ~150k roles, and ~3k groups, how would 
you suggest this update process be approached?

-Patrick Barnes

View raw message