From user-return-15893-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Wed Apr 20 10:38:15 2011 Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 139611A00 for ; Wed, 20 Apr 2011 10:38:15 +0000 (UTC) Received: (qmail 13693 invoked by uid 500); 20 Apr 2011 10:38:13 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 13643 invoked by uid 500); 20 Apr 2011 10:38:13 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 13635 invoked by uid 99); 20 Apr 2011 10:38:12 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Apr 2011 10:38:12 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of paul.hirst@sophos.com designates 74.202.89.161 as permitted sender) Received: from [74.202.89.161] (HELO mx4.sophos.com) (74.202.89.161) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Apr 2011 10:38:07 +0000 Received: from mx4.sophos.com (localhost.localdomain [127.0.0.1]) by localhost (Postfix) with SMTP id 876AE701B6 for ; Wed, 20 Apr 2011 11:37:46 +0100 (BST) Received: from uk-exch2.green.sophos (uk-exch2.green.sophos [10.100.199.17]) by mx4.sophos.com (Postfix) with ESMTP id 264E17018D for ; Wed, 20 Apr 2011 11:37:46 +0100 (BST) Received: from [10.181.0.194] (10.181.0.194) by uk-exch2.green.sophos (10.100.199.10) with Microsoft SMTP Server (TLS) id 8.1.340.0; Wed, 20 Apr 2011 11:37:45 +0100 Subject: Re: Spontaneous view rebuild after view compaction? From: Paul Hirst To: "user@couchdb.apache.org" In-Reply-To: References: <1302700055.15771.14.camel@meerkat.green.sophos> Content-Type: text/plain; charset="UTF-8" Date: Wed, 20 Apr 2011 11:37:43 +0100 Message-ID: <1303295864.15771.88.camel@meerkat.green.sophos> MIME-Version: 1.0 X-Mailer: Evolution 2.28.3 Content-Transfer-Encoding: quoted-printable DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sophos.com; h=subject:from:to:in-reply-to:references:content-type:date:message-id:mime-version:content-transfer-encoding; s=global; bh=Mmtyo1OWfU0emf3jf/EKRHcVCiOSrzZYTwXQrCInEtE=; b=Ogyf7HHf4ebz9MB7DQKPFzoZrS3DsrE0a6DBehMZ8wj6WeNewkvavp0CLSdRmThUEBWM2v8YfnYB93BaIZFpspRefu/jTc9efMkqNuHSM4cMMCwtRDlbquMW9njcagNxSmsUBMkVWsHo+HGUjD1rqHfZxsTgG/DoPL5V59+fD/s= Thanks for all the feedback. I finally managed to get the view to compact successfully. I switched off my automatic view touching script and there were no systems accessing the view. I think that those changes, plus maybe a bit of luck, got it all the way through. After reading the ticket I was tempted to switch off delayed_commits so see if that helped but in the end I didn't need too. Thanks. On Thu, 2011-04-14 at 16:33 +0100, Adam Kocoloski wrote: > Hi Paul, I think you're referring to https://issues.apache.org/jira/brows= e/COUCHDB-994 > > Adam > > On Apr 14, 2011, at 8:13 AM, Cory Zue wrote: > > > Hi Paul, > > > > I encountered a similar issue with my largest view, but it was > > rebuilding whenever couch restarted. I didn't tie it to compaction > > because in my app couch restarts more frequently then we compact, > > however it might have been the same problem. In the end, rewriting the > > view to make the footprint much smaller seemed to resolve it for me, > > but it was a very unsatisfying workaround. > > > > Cory > > > > > > On Thu, Apr 14, 2011 at 4:48 AM, Robert Newson wrote: > >> Hi Paul, > >> > >> I believe this odd behavior has been seen in the wild but I don't > >> think the cause has been isolated. > >> > >> B. > >> > >> On 13 April 2011 14:07, Paul Hirst wrote: > >>> I've had my largest view spontaneously decide to rebuild itself from = scratch a couple of times now, starting immediately after a view compaction= finishes. > >>> > >>> I seem to remember someone pointing at a ticket for this problem a wh= ile ago but searching around in Jira I couldn't find it. Does it ring any b= ells with anyone? > >>> > >>> I have a suspicion I'm hitting some kind of race condition whereby if= the view is queried while the compaction process is attempting to swap the= files over it just triggers a fresh build. I've turned off my regular view= touching for the moment and I'm going to see if it happens again but as my= view takes over a day to build it's getting pretty frustrating so any sugg= estions would be really appreciated. > >>> > >>> Thanks. > >>> > >>> > >>> ________________________________ > >>> Sophos Limited, The Pentagon, Abingdon Science Park, Abingdon, OX14 3= YP, United Kingdom. > >>> Company Reg No 2096520. VAT Reg No GB 991 2418 08. > >>> > >> > Sophos Limited, The Pentagon, Abingdon Science Park, Abingdon, OX14 3YP, Un= ited Kingdom. Company Reg No 2096520. VAT Reg No GB 991 2418 08.