From user-return-19255-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Mon Dec 26 18:11:59 2011 Return-Path: X-Original-To: apmail-couchdb-user-archive@www.apache.org Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 075FD9A26 for ; Mon, 26 Dec 2011 18:11:59 +0000 (UTC) Received: (qmail 40084 invoked by uid 500); 26 Dec 2011 18:11:57 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 40054 invoked by uid 500); 26 Dec 2011 18:11:57 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 40046 invoked by uid 99); 26 Dec 2011 18:11:57 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Dec 2011 18:11:57 +0000 X-ASF-Spam-Status: No, hits=-0.0 required=5.0 tests=RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.210.180] (HELO mail-iy0-f180.google.com) (209.85.210.180) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 26 Dec 2011 18:11:49 +0000 Received: by iazz13 with SMTP id z13so21478203iaz.11 for ; Mon, 26 Dec 2011 10:11:28 -0800 (PST) Received: by 10.42.161.10 with SMTP id r10mr26637563icx.22.1324923088462; Mon, 26 Dec 2011 10:11:28 -0800 (PST) Received: from postel.my-domain ([38.109.25.246]) by mx.google.com with ESMTPS id d19sm80019090ibh.8.2011.12.26.10.11.26 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 26 Dec 2011 10:11:27 -0800 (PST) Content-Type: text/plain; charset=iso-8859-1 Mime-Version: 1.0 (Apple Message framework v1251.1) Subject: Re: Database size seems off even after compaction runs. From: Nathan Vander Wilt In-Reply-To: Date: Mon, 26 Dec 2011 12:10:54 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: <03DB0C45-C307-48DA-A1B9-407235AABB68@calftrail.com> References: <83E168FD-649E-44D1-B2B1-3E9AE728CE98@couchbase.com> <4EF478AA.1080807@gmail.com> To: user@couchdb.apache.org X-Mailer: Apple Mail (2.1251.1) On Dec 23, 2011, at 2:56 PM, Chris Stockton wrote: > Hello, >=20 > On Fri, Dec 23, 2011 at 5:48 AM, CGS wrote: >> Hi, >>=20 >> Sorry to interfere with such a question, but why don't you work with = a >> buffer database? I mean, make a replica to another database which = filters >> out the deleted documents. In such way you can clean all your = databases and >> you use temporary some extra-space (only during the "cleaning" = process). >> Another idea would be to use two databases: one active and one = inactive at >> the given time. That means, you move the data from one to the other, >> filtering out the deleted documents, and when it's over, you switch = to the >> newly constructed database, while the other gets emptied (deleted and >> re-created). Just my 2c opinions. >>=20 >> CGS >>=20 >=20 > Thanks everyone for the various feedback. Now the information I have > gathered is the disk utilization we are seeing is simply from the > deleted documents. >=20 > The question I have yet to see answered (perhaps because it simply > isn't possible) is how to reclaim this space? One potential option, not sure if it's ideal for your use case, could be = using the purge command on the deleted documents, followed by a = compaction. What this does is *completely forget* the document in the btree (no = "tombstones"): POST /db/_purge -d '{"id1":["rev1"], "id2":["rev2"]}' This will thwart replication, i.e. this deletion will not propagate and = the document may get pushed back into existence if another database has = it. hth, -natevw=