couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Everton <...@iocaine.org>
Subject Re: Bulk insert-or-update
Date Thu, 24 Jan 2013 01:30:09 GMT
You appear to be hitting this bug:

https://issues.apache.org/jira/browse/COUCHDB-1415

I'm not aware of any workaround other than adding a salt to the
document or waiting for it to be fixed in CouchDB.

Cheers,
Dan

On Wed, Jan 23, 2013 at 9:14 AM, Mark Wakabayashi <mark@moshymoshy.com> wrote:
> Hi,
>
> I need a way to automatically ensure that certain documents exist with
> pre-specified content as part of a script. That is, I have a file with a
> set of documents, and I want to run a script that will either insert them
> if they don't exist, or else ensure that the documents in the database
> exactly match what I have in the file. Any documents not specified in the
> file should remain untouched.
>
> The best way I've found so far to do this has been to:
> *have the documents in a file in the format described in
> http://wiki.apache.org/couchdb/HTTP_Bulk_Document_API
> *delete each document. Something like:
> for ID in `grep "_id" $FILE | sed 's/.*"_id":"\([^"]*\)".*/\1/'`; do
>   REV=`curl -X GET --silent http://${COUCH_HOST}/foo/${ID} | sed
> 's/.*"_rev":"\([^"]*\)".*/\1/'`
>   curl -X DELETE http://${COUCH_HOST}/foo/${ID}?rev=${REV}
> done
> *insert the documents as they are in the file using _bulk_docs
>
> Is there a better way to do this?
>
>
> This seems to work most of the time, but I'm having intermittent failures
> where the _bulk_docs reports success but doesn't actually insert the
> documents. If I run the script repeatedly, the bulk insert will
> occasionally report that the documents have been inserted with revision 1,
> and the documents are then reported as 'deleted'. That is, the bulk insert
> returns something like:
> [{"ok":true,"id":"bar","rev":"1-243f8f87ed4b0abe0ef00c725d346e07"},...]
> and document "bar" remains deleted, where normally it would return much
> higher revisions, like
> [{"ok":true,"id":"bar","rev":"229-98ff2bb4ad8754cb254ef2e0392d6ab0"},...]
> and "bar" would be available.
>
> Running the script again fixes the problem. Is this a bug in bulk
> insertion? Are there any relevant limitations of bulk insertion that I
> should be aware of?
>
> Thanks in advance,
> Mark

Mime
View raw message