couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sean Copenhaver <>
Subject Re: data migration / ETL
Date Fri, 21 Oct 2011 14:20:26 GMT
I want to say someone announced a basic utility that safely rolls through
all docs in the database and performs a task. I think it was a node.js
command line tool and you provided it a script to run on each document. Not

If you have a view that gives you all the documents based on a version
attribute, then you could roll through all documents in the view for a
particular version (null, 1.1, etc) perform your ETL process, update the
version, and store them back. Eventually you should have no documents with a
version below the one you are migrating to. Only concern is if you run into
conflict because someone updates the doc while you are migrating it, but I
suppose in this situation you would follow your normal conflict resolution
then migrate the resulting doc.

Anyway just some thoughts off the top of my head. I hope that helps some.

On Tue, Oct 18, 2011 at 9:13 AM, daniel williams

> My organization has run into a handful of issues around migration of
> structures of documents from varying versions of software built upon
> couchdb.  Has anyone in the nets come up with a good solution for
> structural
> changes of documents that coincide with major/minor/maintenance releases of
> software leverage couchdb?  perhaps a couch ETL?
> Thanks.
> dan

“The limits of language are the limits of one's world. “ - Ludwig von

"Water is fluid, soft and yielding. But water will wear away rock, which is
rigid and cannot yield. As a rule, whatever is fluid, soft and yielding will
overcome whatever is rigid and hard. This is another paradox: what is soft
is strong." - Lao-Tzu

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message