couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szabo, Viktor" <Viktor.Sz...@morganstanley.com>
Subject RE: possible compact bug in 1.1.1
Date Mon, 20 Feb 2012 21:16:12 GMT
Hi,

An application we have is using bulk writes to insert objects into the database, and occasionally
some
documents get lost. It turns out this happens for the very same reason described below. If
we're
re-insering a doc using an id that has been deleted earlier, if the contents match the payload
that was
saved when the id got used for the first time, the doc remains in "deleted" state and the
client
gets a misleading response. This only happens when the database is compacted after the delete
operation
is executed.

I still feel that this is pretty nasty - what do you guys think?

Looking at the reply coming from _bulk_docs, there's no way I can tell if I can read back
my doc from the database or not.

Below are the steps to reproduce.

Thanks,
Viktor

$ curl -X POST -H "Content-type: application/json" http://server/database/_bulk_docs -d '{"docs"
: [ { "_id" : "bug", "key" : "value" } ] }'
[{"id":"bug","rev":"1-59414e77c768bc202142ac82c2f129de"}]
 
$ curl -X GET http://server/database/bug
{"_id":"bug","_rev":"1-59414e77c768bc202142ac82c2f129de","key":"value"}
 
$ curl -X DELETE http://server/database/bug?rev=1-59414e77c768bc202142ac82c2f129de
{"ok":true,"id":"bug","rev":"2-9b2e3bcc3752a3a952a3570b2ed4d27e"}
 
$ curl -X GET http://server/database/bug
{"error":"not_found","reason":"deleted"}
 
$ curl -X POST -H "Content-type: application/json" http://server/database/_bulk_docs -d '{"docs"
: [ { "_id" : "bug", "key" : "value" } ] }'
[{"id":"bug","rev":"3-82b9390af5b7c003e03c0dd7e6aac45a"}]
 
$ curl -X GET http://server/database/bug
{"_id":"bug","_rev":"3-82b9390af5b7c003e03c0dd7e6aac45a","key":"value"}
 
$ curl -X DELETE http://server/database/bug?rev=3-82b9390af5b7c003e03c0dd7e6aac45a
{"ok":true,"id":"bug","rev":"4-1f98d35af7fd5ce66197980f295e5dba"}
 
$  curl -X POST -H "Content-type: application/json" http://server/database/_compact
{"ok":true}
 
$ curl -X GET http://server/database/bug
{"error":"not_found","reason":"deleted"}
 
$ curl -X POST -H "Content-type: application/json" http://server/database/_bulk_docs -d '{"docs"
: [ { "_id" : "bug", "key" : "value" } ] }'
[{"id":"bug","rev":"1-59414e77c768bc202142ac82c2f129de"}]
 
$ curl -X GET http://server/database/bug
{"error":"not_found","reason":"deleted"}
 

-----Original Message-----
From: Marcello Nuccio [mailto:marcello.nuccio@gmail.com] 
Sent: 23 November 2011 14:38
To: user@couchdb.apache.org
Subject: Re: possible compact bug in 1.1.1

When a new document is created, the response is 201, not 200.
If the posted document is identical to the one saved, the revision is the same (otherwise
it would have been a conflict). So the revision you have is not obsolete.

Marcello

2011/11/23 Szabo, Viktor (Enterprise Infrastructure)
<Viktor.Szabo@morganstanley.com>:
> I'd rather know about the fact that I haven't just successfully 
> created a doc, but re-submitted a revision that was already known - 
> and is already obsolete as revisions with higher version numbers already exist.
>
> Cheers,
> Viktor
>
> -----Original Message-----
> From: Marcello Nuccio [mailto:marcello.nuccio@gmail.com]
> Sent: 23 November 2011 13:41
> To: user@couchdb.apache.org
> Subject: Re: possible compact bug in 1.1.1
>
> Can you elaborate on why you want a conflict?
> I find it confusing to have a conflict when, in fact, there can't be any conflict since
nothing has changed.
>
> Marcello
>
> 2011/11/23 Szabo, Viktor (Enterprise Infrastructure)
> <Viktor.Szabo@morganstanley.com>:
>> Thanks Paul, this makes sense.
>>
>> If it counts, I vote for forcing a conflict ;)
>>
>> Cheers,
>> Viktor
>>
>> -----Original Message-----
>> From: Paul Davis [mailto:paul.joseph.davis@gmail.com]
>> Sent: 22 November 2011 20:54
>> To: user@couchdb.apache.org
>> Subject: Re: possible compact bug in 1.1.1
>>
>>
>> Your example here is actually hitting a very specific edge case as demonstrated by
Marcello's test. As of many versions ago, revisions are generated using a hashing scheme of
the document contents. In your particular case the requests you're issuing contain the same
identical data in such a way that CouchDB will generate a revision of the doc.
>>
>> Given this, we then have to look at how this plays into replication.
>> Basically, when we merge the revision trees we get to the case where it's "oh, we
already have this version, cool" because we do already have this version.
>>
>> Whether or not that behavior is best, or if we should force a conflict if we don't
add a leaf during a write is another question. In other words, the system is working fine,
but this particular behavior can be a bit unexpected.
>>
>> ---------------------------------------------------------------------
>> -
>> ----
>> NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views
contained herein are not intended to be, and do not constitute, advice within the meaning
of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have
received this communication in error, please destroy all electronic and paper copies and notify
the sender immediately. Mistransmission is not intended to waive confidentiality or privilege.
Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor
electronic communications. This message is subject to terms available at the following link:
http://www.morganstanley.com/disclaimers. If you cannot access these links, please notify
us by reply message and we will send the contents to you. By messaging with Morgan Stanley
you consent to the foregoing.
>
> ----------------------------------------------------------------------
> ----
> NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views
contained herein are not intended to be, and do not constitute, advice within the meaning
of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have
received this communication in error, please destroy all electronic and paper copies and notify
the sender immediately. Mistransmission is not intended to waive confidentiality or privilege.
Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor
electronic communications. This message is subject to terms available at the following link:
http://www.morganstanley.com/disclaimers. If you cannot access these links, please notify
us by reply message and we will send the contents to you. By messaging with Morgan Stanley
you consent to the foregoing.

--------------------------------------------------------------------------
NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained
herein are not intended to be, and do not constitute, advice within the meaning of Section
975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received
this communication in error, please destroy all electronic and paper copies and notify the
sender immediately. Mistransmission is not intended to waive confidentiality or privilege.
Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor
electronic communications. This message is subject to terms available at the following link:
http://www.morganstanley.com/disclaimers. If you cannot access these links, please notify
us by reply message and we will send the contents to you. By messaging with Morgan Stanley
you consent to the foregoing.

Mime
View raw message