incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From CGS <cgsmcml...@gmail.com>
Subject Re: possible compact bug in 1.1.1
Date Mon, 20 Feb 2012 23:33:19 GMT
I would dare another question, if I may. Could you tell me if there is
anything in the CouchDB log which would give a hint that the insertion
wasn't successful? Thank you.

CGS




On Tue, Feb 21, 2012 at 12:17 AM, CGS <cgsmcmlxxv@gmail.com> wrote:

> It does matter because it may be that during compaction, the bulk
> insertion may see no document with that name, so, it reports back the first
> revision for the document (1-...). As far as I know (and I might be wrong
> here), the compaction keeps the tombstone for the deleted document and that
> is a revision for the document. That means, your document should have a
> revision higher than 1-... after the re-insertion because the document
> exists even if marked as deleted. But if you say that happens after the
> compaction was completed, then it is clear the scenario is not the same
> with what I presented here.
>
> In your final insertion, you got the exact first revision as you would
> create it anew. I usually take that as an indicator that something went
> wrong during the insertion. I am not saying it's not a bug.
>
> I am not a developer (just helping occasionally in maintaining the
> documentation), but I try to understand the bugs for the next user to be
> aware of such and how to catch it if the bug is not eliminated. So, my
> question is purely for information and understanding. The rest depends on
> the developers.
>
> Thank you for your understanding and your patience in answering my
> question. I will follow this discussion and its outcome.
>
> CGS
>
>
>
>
>
> On Mon, Feb 20, 2012 at 11:42 PM, Szabo, Viktor <
> Viktor.Szabo@morganstanley.com> wrote:
>
>> Yes, the re-insertion took place after the compaction completed.
>>
>> Should it matter? Does it matter?
>>
>> Thanks,
>> Viktor
>>
>> -----Original Message-----
>> From: CGS [mailto:cgsmcmlxxv@gmail.com]
>> Sent: 20 February 2012 23:22
>> To: user@couchdb.apache.org
>> Subject: Re: possible compact bug in 1.1.1
>>
>> Hi,
>>
>> Was the compaction finished when you tried to insert the document?
>>
>> CGS
>>
>>
>>
>>
>> On Mon, Feb 20, 2012 at 10:16 PM, Szabo, Viktor <
>> Viktor.Szabo@morganstanley.com> wrote:
>>
>> > Hi,
>> >
>> > An application we have is using bulk writes to insert objects into the
>> > database, and occasionally some documents get lost. It turns out this
>> > happens for the very same reason described below. If we're re-insering
>> > a doc using an id that has been deleted earlier, if the contents match
>> > the payload that was saved when the id got used for the first time,
>> > the doc remains in "deleted" state and the client gets a misleading
>> > response. This only happens when the database is compacted after the
>> > delete operation is executed.
>> >
>> > I still feel that this is pretty nasty - what do you guys think?
>> >
>> > Looking at the reply coming from _bulk_docs, there's no way I can tell
>> > if I can read back my doc from the database or not.
>> >
>> > Below are the steps to reproduce.
>> >
>> > Thanks,
>> > Viktor
>> >
>> > $ curl -X POST -H "Content-type: application/json"
>> > http://server/database/_bulk_docs -d '{"docs" : [ { "_id" : "bug",
>> "key"
>> > : "value" } ] }'
>> > [{"id":"bug","rev":"1-59414e77c768bc202142ac82c2f129de"}]
>> >
>> > $ curl -X GET http://server/database/bug
>> > {"_id":"bug","_rev":"1-59414e77c768bc202142ac82c2f129de","key":"value"
>> > }
>> >
>> > $ curl -X DELETE
>> > http://server/database/bug?rev=1-59414e77c768bc202142ac82c2f129de
>> > {"ok":true,"id":"bug","rev":"2-9b2e3bcc3752a3a952a3570b2ed4d27e"}
>> >
>> > $ curl -X GET http://server/database/bug
>> > {"error":"not_found","reason":"deleted"}
>> >
>> > $ curl -X POST -H "Content-type: application/json"
>> > http://server/database/_bulk_docs -d '{"docs" : [ { "_id" : "bug",
>> "key"
>> > : "value" } ] }'
>> > [{"id":"bug","rev":"3-82b9390af5b7c003e03c0dd7e6aac45a"}]
>> >
>> > $ curl -X GET http://server/database/bug
>> > {"_id":"bug","_rev":"3-82b9390af5b7c003e03c0dd7e6aac45a","key":"value"
>> > }
>> >
>> > $ curl -X DELETE
>> > http://server/database/bug?rev=3-82b9390af5b7c003e03c0dd7e6aac45a
>> > {"ok":true,"id":"bug","rev":"4-1f98d35af7fd5ce66197980f295e5dba"}
>> >
>> > $  curl -X POST -H "Content-type: application/json"
>> > http://server/database/_compact
>> > {"ok":true}
>> >
>> > $ curl -X GET http://server/database/bug
>> > {"error":"not_found","reason":"deleted"}
>> >
>> > $ curl -X POST -H "Content-type: application/json"
>> > http://server/database/_bulk_docs -d '{"docs" : [ { "_id" : "bug",
>> "key"
>> > : "value" } ] }'
>> > [{"id":"bug","rev":"1-59414e77c768bc202142ac82c2f129de"}]
>> >
>> > $ curl -X GET http://server/database/bug
>> > {"error":"not_found","reason":"deleted"}
>> >
>> >
>> > -----Original Message-----
>> > From: Marcello Nuccio [mailto:marcello.nuccio@gmail.com]
>> > Sent: 23 November 2011 14:38
>> > To: user@couchdb.apache.org
>> > Subject: Re: possible compact bug in 1.1.1
>> >
>> > When a new document is created, the response is 201, not 200.
>> > If the posted document is identical to the one saved, the revision is
>> > the same (otherwise it would have been a conflict). So the revision
>> > you have is not obsolete.
>> >
>> > Marcello
>> >
>> > 2011/11/23 Szabo, Viktor (Enterprise Infrastructure)
>> > <Viktor.Szabo@morganstanley.com>:
>> > > I'd rather know about the fact that I haven't just successfully
>> > > created a doc, but re-submitted a revision that was already known -
>> > > and is already obsolete as revisions with higher version numbers
>> > > already
>> > exist.
>> > >
>> > > Cheers,
>> > > Viktor
>> > >
>> > > -----Original Message-----
>> > > From: Marcello Nuccio [mailto:marcello.nuccio@gmail.com]
>> > > Sent: 23 November 2011 13:41
>> > > To: user@couchdb.apache.org
>> > > Subject: Re: possible compact bug in 1.1.1
>> > >
>> > > Can you elaborate on why you want a conflict?
>> > > I find it confusing to have a conflict when, in fact, there can't be
>> > > any
>> > conflict since nothing has changed.
>> > >
>> > > Marcello
>> > >
>> > > 2011/11/23 Szabo, Viktor (Enterprise Infrastructure)
>> > > <Viktor.Szabo@morganstanley.com>:
>> > >> Thanks Paul, this makes sense.
>> > >>
>> > >> If it counts, I vote for forcing a conflict ;)
>> > >>
>> > >> Cheers,
>> > >> Viktor
>> > >>
>> > >> -----Original Message-----
>> > >> From: Paul Davis [mailto:paul.joseph.davis@gmail.com]
>> > >> Sent: 22 November 2011 20:54
>> > >> To: user@couchdb.apache.org
>> > >> Subject: Re: possible compact bug in 1.1.1
>> > >>
>> > >>
>> > >> Your example here is actually hitting a very specific edge case as
>> > demonstrated by Marcello's test. As of many versions ago, revisions
>> > are generated using a hashing scheme of the document contents. In your
>> > particular case the requests you're issuing contain the same identical
>> > data in such a way that CouchDB will generate a revision of the doc.
>> > >>
>> > >> Given this, we then have to look at how this plays into replication.
>> > >> Basically, when we merge the revision trees we get to the case
>> > >> where
>> > it's "oh, we already have this version, cool" because we do already
>> > have this version.
>> > >>
>> > >> Whether or not that behavior is best, or if we should force a
>> > >> conflict
>> > if we don't add a leaf during a write is another question. In other
>> > words, the system is working fine, but this particular behavior can be
>> > a bit unexpected.
>> > >>
>> > >> -------------------------------------------------------------------
>> > >> --
>> > >> -
>> > >> ----
>> > >> NOTICE: Morgan Stanley is not acting as a municipal advisor and the
>> > opinions or views contained herein are not intended to be, and do not
>> > constitute, advice within the meaning of Section 975 of the Dodd-Frank
>> > Wall Street Reform and Consumer Protection Act. If you have received
>> > this communication in error, please destroy all electronic and paper
>> > copies and notify the sender immediately. Mistransmission is not
>> > intended to waive confidentiality or privilege. Morgan Stanley
>> > reserves the right, to the extent permitted under applicable law, to
>> > monitor electronic communications. This message is subject to terms
>> > available at the following
>> > link: http://www.morganstanley.com/disclaimers. If you cannot access
>> > these links, please notify us by reply message and we will send the
>> > contents to you. By messaging with Morgan Stanley you consent to the
>> > foregoing.
>> > >
>> > > --------------------------------------------------------------------
>> > > --
>> > > ----
>> > > NOTICE: Morgan Stanley is not acting as a municipal advisor and the
>> > opinions or views contained herein are not intended to be, and do not
>> > constitute, advice within the meaning of Section 975 of the Dodd-Frank
>> > Wall Street Reform and Consumer Protection Act. If you have received
>> > this communication in error, please destroy all electronic and paper
>> > copies and notify the sender immediately. Mistransmission is not
>> > intended to waive confidentiality or privilege. Morgan Stanley
>> > reserves the right, to the extent permitted under applicable law, to
>> > monitor electronic communications. This message is subject to terms
>> > available at the following
>> > link: http://www.morganstanley.com/disclaimers. If you cannot access
>> > these links, please notify us by reply message and we will send the
>> > contents to you. By messaging with Morgan Stanley you consent to the
>> > foregoing.
>> >
>> > ----------------------------------------------------------------------
>> > ----
>> > NOTICE: Morgan Stanley is not acting as a municipal advisor and the
>> > opinions or views contained herein are not intended to be, and do not
>> > constitute, advice within the meaning of Section 975 of the Dodd-Frank
>> > Wall Street Reform and Consumer Protection Act. If you have received
>> > this communication in error, please destroy all electronic and paper
>> > copies and notify the sender immediately. Mistransmission is not
>> > intended to waive confidentiality or privilege. Morgan Stanley
>> > reserves the right, to the extent permitted under applicable law, to
>> > monitor electronic communications. This message is subject to terms
>> > available at the following
>> > link: http://www.morganstanley.com/disclaimers. If you cannot access
>> > these links, please notify us by reply message and we will send the
>> > contents to you. By messaging with Morgan Stanley you consent to the
>> > foregoing.
>> >
>>
>> --------------------------------------------------------------------------
>> NOTICE: Morgan Stanley is not acting as a municipal advisor and the
>> opinions or views contained herein are not intended to be, and do not
>> constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall
>> Street Reform and Consumer Protection Act. If you have received this
>> communication in error, please destroy all electronic and paper copies and
>> notify the sender immediately. Mistransmission is not intended to waive
>> confidentiality or privilege. Morgan Stanley reserves the right, to the
>> extent permitted under applicable law, to monitor electronic
>> communications. This message is subject to terms available at the following
>> link: http://www.morganstanley.com/disclaimers. If you cannot access
>> these links, please notify us by reply message and we will send the
>> contents to you. By messaging with Morgan Stanley you consent to the
>> foregoing.
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message