incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Ramage <ryan.ram...@gmail.com>
Subject Re: data migration / ETL
Date Fri, 21 Oct 2011 14:46:51 GMT
There is Costco:
https://github.com/harthur/costco

Or Recline:
https://github.com/maxogden/recline

Not sure if those will have enough horsepower for you though.

After reading your question again, it seems to be more about release
management...the versions of software using docs in couch, and
managing the document 'schema'.  In the sql world I have used liqibase
[http://www.liquibase.org/manual/home]. Although I find schema-less
much more relaxing,  I could see some cases where a tool like
liquibase for couchdb would be useful.

Ryan


On Fri, Oct 21, 2011 at 8:20 AM, Sean Copenhaver
<sean.copenhaver@gmail.com> wrote:
> I want to say someone announced a basic utility that safely rolls through
> all docs in the database and performs a task. I think it was a node.js
> command line tool and you provided it a script to run on each document. Not
> positive.
>
> If you have a view that gives you all the documents based on a version
> attribute, then you could roll through all documents in the view for a
> particular version (null, 1.1, etc) perform your ETL process, update the
> version, and store them back. Eventually you should have no documents with a
> version below the one you are migrating to. Only concern is if you run into
> conflict because someone updates the doc while you are migrating it, but I
> suppose in this situation you would follow your normal conflict resolution
> then migrate the resulting doc.
>
> Anyway just some thoughts off the top of my head. I hope that helps some.
>
> On Tue, Oct 18, 2011 at 9:13 AM, daniel williams
> <daniel.williams@gmail.com>wrote:
>
>> My organization has run into a handful of issues around migration of
>> structures of documents from varying versions of software built upon
>> couchdb.  Has anyone in the nets come up with a good solution for
>> structural
>> changes of documents that coincide with major/minor/maintenance releases of
>> software leverage couchdb?  perhaps a couch ETL?
>>
>> Thanks.
>>
>> dan
>>
>
>
>
> --
> “The limits of language are the limits of one's world. “ - Ludwig von
> Wittgenstein
>
> "Water is fluid, soft and yielding. But water will wear away rock, which is
> rigid and cannot yield. As a rule, whatever is fluid, soft and yielding will
> overcome whatever is rigid and hard. This is another paradox: what is soft
> is strong." - Lao-Tzu
>


-- 
Twitter: @eckoit

Mime
View raw message