From dev-return-13930-apmail-couchdb-dev-archive=couchdb.apache.org@couchdb.apache.org Tue Dec 28 12:21:10 2010 Return-Path: Delivered-To: apmail-couchdb-dev-archive@www.apache.org Received: (qmail 19548 invoked from network); 28 Dec 2010 12:21:09 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 28 Dec 2010 12:21:09 -0000 Received: (qmail 36768 invoked by uid 500); 28 Dec 2010 12:21:09 -0000 Delivered-To: apmail-couchdb-dev-archive@couchdb.apache.org Received: (qmail 36576 invoked by uid 500); 28 Dec 2010 12:21:09 -0000 Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@couchdb.apache.org Delivered-To: mailing list dev@couchdb.apache.org Received: (qmail 36568 invoked by uid 99); 28 Dec 2010 12:21:08 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Dec 2010 12:21:08 +0000 X-ASF-Spam-Status: No, hits=-2000.0 required=10.0 tests=ALL_TRUSTED X-Spam-Check-By: apache.org Received: from [140.211.11.22] (HELO thor.apache.org) (140.211.11.22) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 28 Dec 2010 12:21:07 +0000 Received: from thor (localhost [127.0.0.1]) by thor.apache.org (8.13.8+Sun/8.13.8) with ESMTP id oBSCKks7023235 for ; Tue, 28 Dec 2010 12:20:47 GMT Message-ID: <3208653.42671293538846883.JavaMail.jira@thor> Date: Tue, 28 Dec 2010 07:20:46 -0500 (EST) From: "Simon Lucy (JIRA)" To: dev@couchdb.apache.org Subject: [jira] Commented: (COUCHDB-968) Duplicated IDs in _all_docs In-Reply-To: <12364317.322241290769453883.JavaMail.jira@thor> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/COUCHDB-968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12975484#action_12975484 ] Simon Lucy commented on COUCHDB-968: ------------------------------------ Is there a summary available somewhere as to where we are with this? > Duplicated IDs in _all_docs > --------------------------- > > Key: COUCHDB-968 > URL: https://issues.apache.org/jira/browse/COUCHDB-968 > Project: CouchDB > Issue Type: Bug > Components: Database Core > Affects Versions: 0.10.1, 0.10.2, 0.11.1, 0.11.2, 1.0, 1.0.1, 1.0.2 > Environment: any > Reporter: Sebastian Cohnen > Assignee: Adam Kocoloski > Priority: Blocker > Fix For: 0.11.3, 1.0.2, 1.1 > > > We have a database, which is causing serious trouble with compaction and replication (huge memory and cpu usage, often causing couchdb to crash b/c all system memory is exhausted). Yesterday we discovered that db/_all_docs is reporting duplicated IDs (see [1]). Until a few minutes ago we thought that there are only few duplicates but today I took a closer look and I found 10 IDs which sum up to a total of 922 duplicates. Some of them have only 1 duplicate, others have hundreds. > Some facts about the database in question: > * ~13k documents, with 3-5k revs each > * all duplicated documents are in conflict (with 1 up to 14 conflicts) > * compaction is run on a daily bases > * several thousands updates per hour > * multi-master setup with pull replication from each other > * delayed_commits=false on all nodes > * used couchdb versions 1.0.0 and 1.0.x (*) > Unfortunately the database's contents are confidential and I'm not allowed to publish it. > [1]: Part of http://localhost:5984/DBNAME/_all_docs > ... > {"id":"9997","key":"9997","value":{"rev":"6096-603c68c1fa90ac3f56cf53771337ac9f"}}, > {"id":"9999","key":"9999","value":{"rev":"6097-3c873ccf6875ff3c4e2c6fa264c6a180"}}, > {"id":"9999","key":"9999","value":{"rev":"6097-3c873ccf6875ff3c4e2c6fa264c6a180"}}, > ... > [*] > There were two (old) servers (1.0.0) in production (already having the replication and compaction issues). Then two servers (1.0.x) were added and replication was set up to bring them in sync with the old production servers since the two new servers were meant to replace the old ones (to update node.js application code among other things). -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.