From user-return-19878-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Mon Feb 20 23:33:50 2012 Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 2B9C89FDC for ; Mon, 20 Feb 2012 23:33:50 +0000 (UTC) Received: (qmail 91318 invoked by uid 500); 20 Feb 2012 23:33:48 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 91288 invoked by uid 500); 20 Feb 2012 23:33:48 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 91280 invoked by uid 99); 20 Feb 2012 23:33:48 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 20 Feb 2012 23:33:48 +0000 X-ASF-Spam-Status: No, hits=3.6 required=5.0 tests=FROM_LOCAL_NOVOWEL,FSL_RCVD_USER,HK_RANDOM_ENVFROM,HK_RANDOM_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of cgsmcmlxxv@gmail.com designates 209.85.216.45 as permitted sender) Received: from [209.85.216.45] (HELO mail-qw0-f45.google.com) (209.85.216.45) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 20 Feb 2012 23:33:40 +0000 Received: by qabg40 with SMTP id g40so4098298qab.11 for ; Mon, 20 Feb 2012 15:33:20 -0800 (PST) Received-SPF: pass (google.com: domain of cgsmcmlxxv@gmail.com designates 10.229.102.87 as permitted sender) client-ip=10.229.102.87; Authentication-Results: mr.google.com; spf=pass (google.com: domain of cgsmcmlxxv@gmail.com designates 10.229.102.87 as permitted sender) smtp.mail=cgsmcmlxxv@gmail.com; dkim=pass header.i=cgsmcmlxxv@gmail.com Received: from mr.google.com ([10.229.102.87]) by 10.229.102.87 with SMTP id f23mr17300014qco.125.1329780800219 (num_hops = 1); Mon, 20 Feb 2012 15:33:20 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=el6SCxgt0UGx2LGUZAemyo671BCO2/J84wFKlwp+JEk=; b=HCw1T36wF5DY57XedWDxUX2tPZfZKEJXcqSaIdvun+OXNpEL7uNTQ3vpI8fTNtbos5 x2z+RBmacjV2BCbDsM6k5azv6tWXd5rjfQIJ6xX7vkK0owXUx1GBddy8DnVjwLWj3rT7 8iYfIcJ42mGxMxvp4VQt+LwOUL3XPsLOCdsFc= MIME-Version: 1.0 Received: by 10.229.102.87 with SMTP id f23mr14673629qco.125.1329780800080; Mon, 20 Feb 2012 15:33:20 -0800 (PST) Received: by 10.229.85.7 with HTTP; Mon, 20 Feb 2012 15:33:19 -0800 (PST) In-Reply-To: References: <9032450E6B3D8244858652FFFD6E29BC0294FC@OZWEX0202N2.msad.ms.com> <9032450E6B3D8244858652FFFD6E29BC02A825@OZWEX0202N2.msad.ms.com> <9032450E6B3D8244858652FFFD6E29BC02A8A0@OZWEX0202N2.msad.ms.com> <9032450E6B3D8244858652FFFD6E29BC10491A@OZWEX0202N2.msad.ms.com> <9032450E6B3D8244858652FFFD6E29BC104B03@OZWEX0202N2.msad.ms.com> Date: Tue, 21 Feb 2012 00:33:19 +0100 Message-ID: Subject: Re: possible compact bug in 1.1.1 From: CGS To: user@couchdb.apache.org Content-Type: multipart/alternative; boundary=00235447139837cc8204b96db811 X-Virus-Checked: Checked by ClamAV on apache.org --00235447139837cc8204b96db811 Content-Type: text/plain; charset=ISO-8859-1 I would dare another question, if I may. Could you tell me if there is anything in the CouchDB log which would give a hint that the insertion wasn't successful? Thank you. CGS On Tue, Feb 21, 2012 at 12:17 AM, CGS wrote: > It does matter because it may be that during compaction, the bulk > insertion may see no document with that name, so, it reports back the first > revision for the document (1-...). As far as I know (and I might be wrong > here), the compaction keeps the tombstone for the deleted document and that > is a revision for the document. That means, your document should have a > revision higher than 1-... after the re-insertion because the document > exists even if marked as deleted. But if you say that happens after the > compaction was completed, then it is clear the scenario is not the same > with what I presented here. > > In your final insertion, you got the exact first revision as you would > create it anew. I usually take that as an indicator that something went > wrong during the insertion. I am not saying it's not a bug. > > I am not a developer (just helping occasionally in maintaining the > documentation), but I try to understand the bugs for the next user to be > aware of such and how to catch it if the bug is not eliminated. So, my > question is purely for information and understanding. The rest depends on > the developers. > > Thank you for your understanding and your patience in answering my > question. I will follow this discussion and its outcome. > > CGS > > > > > > On Mon, Feb 20, 2012 at 11:42 PM, Szabo, Viktor < > Viktor.Szabo@morganstanley.com> wrote: > >> Yes, the re-insertion took place after the compaction completed. >> >> Should it matter? Does it matter? >> >> Thanks, >> Viktor >> >> -----Original Message----- >> From: CGS [mailto:cgsmcmlxxv@gmail.com] >> Sent: 20 February 2012 23:22 >> To: user@couchdb.apache.org >> Subject: Re: possible compact bug in 1.1.1 >> >> Hi, >> >> Was the compaction finished when you tried to insert the document? >> >> CGS >> >> >> >> >> On Mon, Feb 20, 2012 at 10:16 PM, Szabo, Viktor < >> Viktor.Szabo@morganstanley.com> wrote: >> >> > Hi, >> > >> > An application we have is using bulk writes to insert objects into the >> > database, and occasionally some documents get lost. It turns out this >> > happens for the very same reason described below. If we're re-insering >> > a doc using an id that has been deleted earlier, if the contents match >> > the payload that was saved when the id got used for the first time, >> > the doc remains in "deleted" state and the client gets a misleading >> > response. This only happens when the database is compacted after the >> > delete operation is executed. >> > >> > I still feel that this is pretty nasty - what do you guys think? >> > >> > Looking at the reply coming from _bulk_docs, there's no way I can tell >> > if I can read back my doc from the database or not. >> > >> > Below are the steps to reproduce. >> > >> > Thanks, >> > Viktor >> > >> > $ curl -X POST -H "Content-type: application/json" >> > http://server/database/_bulk_docs -d '{"docs" : [ { "_id" : "bug", >> "key" >> > : "value" } ] }' >> > [{"id":"bug","rev":"1-59414e77c768bc202142ac82c2f129de"}] >> > >> > $ curl -X GET http://server/database/bug >> > {"_id":"bug","_rev":"1-59414e77c768bc202142ac82c2f129de","key":"value" >> > } >> > >> > $ curl -X DELETE >> > http://server/database/bug?rev=1-59414e77c768bc202142ac82c2f129de >> > {"ok":true,"id":"bug","rev":"2-9b2e3bcc3752a3a952a3570b2ed4d27e"} >> > >> > $ curl -X GET http://server/database/bug >> > {"error":"not_found","reason":"deleted"} >> > >> > $ curl -X POST -H "Content-type: application/json" >> > http://server/database/_bulk_docs -d '{"docs" : [ { "_id" : "bug", >> "key" >> > : "value" } ] }' >> > [{"id":"bug","rev":"3-82b9390af5b7c003e03c0dd7e6aac45a"}] >> > >> > $ curl -X GET http://server/database/bug >> > {"_id":"bug","_rev":"3-82b9390af5b7c003e03c0dd7e6aac45a","key":"value" >> > } >> > >> > $ curl -X DELETE >> > http://server/database/bug?rev=3-82b9390af5b7c003e03c0dd7e6aac45a >> > {"ok":true,"id":"bug","rev":"4-1f98d35af7fd5ce66197980f295e5dba"} >> > >> > $ curl -X POST -H "Content-type: application/json" >> > http://server/database/_compact >> > {"ok":true} >> > >> > $ curl -X GET http://server/database/bug >> > {"error":"not_found","reason":"deleted"} >> > >> > $ curl -X POST -H "Content-type: application/json" >> > http://server/database/_bulk_docs -d '{"docs" : [ { "_id" : "bug", >> "key" >> > : "value" } ] }' >> > [{"id":"bug","rev":"1-59414e77c768bc202142ac82c2f129de"}] >> > >> > $ curl -X GET http://server/database/bug >> > {"error":"not_found","reason":"deleted"} >> > >> > >> > -----Original Message----- >> > From: Marcello Nuccio [mailto:marcello.nuccio@gmail.com] >> > Sent: 23 November 2011 14:38 >> > To: user@couchdb.apache.org >> > Subject: Re: possible compact bug in 1.1.1 >> > >> > When a new document is created, the response is 201, not 200. >> > If the posted document is identical to the one saved, the revision is >> > the same (otherwise it would have been a conflict). So the revision >> > you have is not obsolete. >> > >> > Marcello >> > >> > 2011/11/23 Szabo, Viktor (Enterprise Infrastructure) >> > : >> > > I'd rather know about the fact that I haven't just successfully >> > > created a doc, but re-submitted a revision that was already known - >> > > and is already obsolete as revisions with higher version numbers >> > > already >> > exist. >> > > >> > > Cheers, >> > > Viktor >> > > >> > > -----Original Message----- >> > > From: Marcello Nuccio [mailto:marcello.nuccio@gmail.com] >> > > Sent: 23 November 2011 13:41 >> > > To: user@couchdb.apache.org >> > > Subject: Re: possible compact bug in 1.1.1 >> > > >> > > Can you elaborate on why you want a conflict? >> > > I find it confusing to have a conflict when, in fact, there can't be >> > > any >> > conflict since nothing has changed. >> > > >> > > Marcello >> > > >> > > 2011/11/23 Szabo, Viktor (Enterprise Infrastructure) >> > > : >> > >> Thanks Paul, this makes sense. >> > >> >> > >> If it counts, I vote for forcing a conflict ;) >> > >> >> > >> Cheers, >> > >> Viktor >> > >> >> > >> -----Original Message----- >> > >> From: Paul Davis [mailto:paul.joseph.davis@gmail.com] >> > >> Sent: 22 November 2011 20:54 >> > >> To: user@couchdb.apache.org >> > >> Subject: Re: possible compact bug in 1.1.1 >> > >> >> > >> >> > >> Your example here is actually hitting a very specific edge case as >> > demonstrated by Marcello's test. As of many versions ago, revisions >> > are generated using a hashing scheme of the document contents. In your >> > particular case the requests you're issuing contain the same identical >> > data in such a way that CouchDB will generate a revision of the doc. >> > >> >> > >> Given this, we then have to look at how this plays into replication. >> > >> Basically, when we merge the revision trees we get to the case >> > >> where >> > it's "oh, we already have this version, cool" because we do already >> > have this version. >> > >> >> > >> Whether or not that behavior is best, or if we should force a >> > >> conflict >> > if we don't add a leaf during a write is another question. In other >> > words, the system is working fine, but this particular behavior can be >> > a bit unexpected. >> > >> >> > >> ------------------------------------------------------------------- >> > >> -- >> > >> - >> > >> ---- >> > >> NOTICE: Morgan Stanley is not acting as a municipal advisor and the >> > opinions or views contained herein are not intended to be, and do not >> > constitute, advice within the meaning of Section 975 of the Dodd-Frank >> > Wall Street Reform and Consumer Protection Act. If you have received >> > this communication in error, please destroy all electronic and paper >> > copies and notify the sender immediately. Mistransmission is not >> > intended to waive confidentiality or privilege. Morgan Stanley >> > reserves the right, to the extent permitted under applicable law, to >> > monitor electronic communications. This message is subject to terms >> > available at the following >> > link: http://www.morganstanley.com/disclaimers. If you cannot access >> > these links, please notify us by reply message and we will send the >> > contents to you. By messaging with Morgan Stanley you consent to the >> > foregoing. >> > > >> > > -------------------------------------------------------------------- >> > > -- >> > > ---- >> > > NOTICE: Morgan Stanley is not acting as a municipal advisor and the >> > opinions or views contained herein are not intended to be, and do not >> > constitute, advice within the meaning of Section 975 of the Dodd-Frank >> > Wall Street Reform and Consumer Protection Act. If you have received >> > this communication in error, please destroy all electronic and paper >> > copies and notify the sender immediately. Mistransmission is not >> > intended to waive confidentiality or privilege. Morgan Stanley >> > reserves the right, to the extent permitted under applicable law, to >> > monitor electronic communications. This message is subject to terms >> > available at the following >> > link: http://www.morganstanley.com/disclaimers. If you cannot access >> > these links, please notify us by reply message and we will send the >> > contents to you. By messaging with Morgan Stanley you consent to the >> > foregoing. >> > >> > ---------------------------------------------------------------------- >> > ---- >> > NOTICE: Morgan Stanley is not acting as a municipal advisor and the >> > opinions or views contained herein are not intended to be, and do not >> > constitute, advice within the meaning of Section 975 of the Dodd-Frank >> > Wall Street Reform and Consumer Protection Act. If you have received >> > this communication in error, please destroy all electronic and paper >> > copies and notify the sender immediately. Mistransmission is not >> > intended to waive confidentiality or privilege. Morgan Stanley >> > reserves the right, to the extent permitted under applicable law, to >> > monitor electronic communications. This message is subject to terms >> > available at the following >> > link: http://www.morganstanley.com/disclaimers. If you cannot access >> > these links, please notify us by reply message and we will send the >> > contents to you. By messaging with Morgan Stanley you consent to the >> > foregoing. >> > >> >> -------------------------------------------------------------------------- >> NOTICE: Morgan Stanley is not acting as a municipal advisor and the >> opinions or views contained herein are not intended to be, and do not >> constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall >> Street Reform and Consumer Protection Act. If you have received this >> communication in error, please destroy all electronic and paper copies and >> notify the sender immediately. Mistransmission is not intended to waive >> confidentiality or privilege. Morgan Stanley reserves the right, to the >> extent permitted under applicable law, to monitor electronic >> communications. This message is subject to terms available at the following >> link: http://www.morganstanley.com/disclaimers. If you cannot access >> these links, please notify us by reply message and we will send the >> contents to you. By messaging with Morgan Stanley you consent to the >> foregoing. >> > > --00235447139837cc8204b96db811--