Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: neutral (athena.apache.org: local policy)
Content-Type: text/plain; charset=us-ascii
Mime-Version: 1.0 (Apple Message framework v1082)
Subject: Re: [jira] Updated: (COUCHDB-968) Duplicated IDs in _all_docs
From: Jan Lehnardt <jan@apache.org>
In-Reply-To: <1200970.2551290897937827.JavaMail.jira@thor>
Date: Sun, 28 Nov 2010 13:55:17 +0100
Content-Transfer-Encoding: quoted-printable
Message-Id: <7CA14791-C5C3-435A-84C5-E7EB28872A09@apache.org>
References: <1200970.2551290897937827.JavaMail.jira@thor>
To: dev@couchdb.apache.org


On 27 Nov 2010, at 23:45, Adam Kocoloski (JIRA) wrote:

>=20
>     [ =
https://issues.apache.org/jira/browse/COUCHDB-968?page=3Dcom.atlassian.jir=
a.plugin.system.issuetabpanels:all-tabpanel ]
>=20
> Adam Kocoloski updated COUCHDB-968:
> -----------------------------------
>=20
>    Priority: Blocker  (was: Major)

Should we hold 1.0.2 for this?

Cheers
Jan
--=20

>=20
> Bob, in tisba's case the duplicates had the same revision.  Is that =
also true in your case?  And you only see these duplicates after =
compaction?
>=20
>> Duplicated IDs in _all_docs
>> ---------------------------
>>=20
>>                Key: COUCHDB-968
>>                URL: https://issues.apache.org/jira/browse/COUCHDB-968
>>            Project: CouchDB
>>         Issue Type: Bug
>>         Components: Database Core
>>   Affects Versions: 1.0, 1.0.1, 1.0.2
>>        Environment: Ubuntu 10.04.
>>           Reporter: Sebastian Cohnen
>>           Priority: Blocker
>>=20
>> We have a database, which is causing serious trouble with compaction =
and replication (huge memory and cpu usage, often causing couchdb to =
crash b/c all system memory is exhausted). Yesterday we discovered that =
db/_all_docs is reporting duplicated IDs (see [1]). Until a few minutes =
ago we thought that there are only few duplicates but today I took a =
closer look and I found 10 IDs which sum up to a total of 922 =
duplicates. Some of them have only 1 duplicate, others have hundreds.
>> Some facts about the database in question:
>> * ~13k documents, with 3-5k revs each
>> * all duplicated documents are in conflict (with 1 up to 14 =
conflicts)
>> * compaction is run on a daily bases
>> * several thousands updates per hour
>> * multi-master setup with pull replication from each other
>> * delayed_commits=3Dfalse on all nodes
>> * used couchdb versions 1.0.0 and 1.0.x (*)
>> Unfortunately the database's contents are confidential and I'm not =
allowed to publish it.
>> [1]: Part of http://localhost:5984/DBNAME/_all_docs
>> ...
>> =
{"id":"9997","key":"9997","value":{"rev":"6096-603c68c1fa90ac3f56cf5377133=
7ac9f"}},
>> =
{"id":"9999","key":"9999","value":{"rev":"6097-3c873ccf6875ff3c4e2c6fa264c=
6a180"}},
>> =
{"id":"9999","key":"9999","value":{"rev":"6097-3c873ccf6875ff3c4e2c6fa264c=
6a180"}},
>> ...
>> [*]
>> There were two (old) servers (1.0.0) in production (already having =
the replication and compaction issues). Then two servers (1.0.x) were =
added and replication was set up to bring them in sync with the old =
production servers since the two new servers were meant to replace the =
old ones (to update node.js application code among other things).
>=20
> --=20
> This message is automatically generated by JIRA.
> -
> You can reply to this email to add a comment to the issue online.
>=20