From user-return-12522-apmail-couchdb-user-archive=couchdb.apache.org@couchdb.apache.org Fri Sep 10 11:25:59 2010 Return-Path: Delivered-To: apmail-couchdb-user-archive@www.apache.org Received: (qmail 40835 invoked from network); 10 Sep 2010 11:25:59 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 10 Sep 2010 11:25:59 -0000 Received: (qmail 16807 invoked by uid 500); 10 Sep 2010 11:25:57 -0000 Delivered-To: apmail-couchdb-user-archive@couchdb.apache.org Received: (qmail 16587 invoked by uid 500); 10 Sep 2010 11:25:55 -0000 Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@couchdb.apache.org Delivered-To: mailing list user@couchdb.apache.org Received: (qmail 16579 invoked by uid 99); 10 Sep 2010 11:25:54 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Sep 2010 11:25:54 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of emargee@gmail.com designates 209.85.212.52 as permitted sender) Received: from [209.85.212.52] (HELO mail-vw0-f52.google.com) (209.85.212.52) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 10 Sep 2010 11:25:47 +0000 Received: by vws14 with SMTP id 14so3222939vws.11 for ; Fri, 10 Sep 2010 04:25:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:received:in-reply-to :references:date:message-id:subject:from:to:content-type; bh=xelOck9NfdqcO1yQPxci7s1sFopV7V6XM1UATKOkuwE=; b=gqBEpHXbnx1V8xO9RQXWOo30Fsi8QdE1QRDITjBhxpHo9Zgw8gja6e4m75B21aqQ8o ngpGgasFqvJvpknE4iSBJJ7AGAzXyqO23GT/j0oLnMSs3MsT1bD/I5dORSMSdTXzb9uS LaGEkH8cs8A8+XmL2FrnJhOanPVa8G8DfUwo0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; b=BLnlcSX60iyFt8X8ZKzE4mfiofK7A4FhyaDlu3ND2YbtCZGX/5pPt0E+brID7l8tW6 1sUqG1vCEB32v5MwbsrOGiUwzZVv7wt4bjTBD8vd629i1C3APtWE56u681C4qmamT2Ol mt9Uw8n8xRp6eq3UB2rPoWJZiMXkfXY4p6Vss= MIME-Version: 1.0 Received: by 10.220.124.85 with SMTP id t21mr369433vcr.145.1284117923444; Fri, 10 Sep 2010 04:25:23 -0700 (PDT) Received: by 10.220.202.11 with HTTP; Fri, 10 Sep 2010 04:25:23 -0700 (PDT) In-Reply-To: <5EC43ABE2BBBA44CA91D3608BDE011C90C4FE58EC2@MAILBGO.vizrt.internal> References: <5EC43ABE2BBBA44CA91D3608BDE011C90C4FE58EC2@MAILBGO.vizrt.internal> Date: Fri, 10 Sep 2010 12:25:23 +0100 Message-ID: Subject: Re: Couch not releasing deleted files From: "[mRg]" To: user@couchdb.apache.org Content-Type: multipart/alternative; boundary=001636c92cd2ad38bb048fe600fa X-Virus-Checked: Checked by ClamAV on apache.org --001636c92cd2ad38bb048fe600fa Content-Type: text/plain; charset=ISO-8859-1 I also tested by replicating to a CouchDB 1.0.1 instance and the same thing occurs (although now the unreleased files are in the .deleted directory). I can reproduce this by the following steps: 1> Load a couchdb instance with dummy data 2> Execute a view 3> Run a compact / cleanup on the view 4> Check lsof on the .deleted directory While 'du' reports its is gone 'df' shows the disk still in use. I thought it may be because of some delay in cleaning the deleted nodes but a day or 2 later that are still there. 5> Restart couchdb and the files are removed. @Panop - The only way I've found is to restart the couchdb service. Regards Stephen On 10 September 2010 10:00, Panop Suvaphrom wrote: > Yeah, me too. On couch 1.0.1 with ubuntu 10.04 > Do you have any idea to operate on this ? > > > > > -----Original Message----- > From: [mRg] [mailto:emargee@gmail.com] > Sent: Thursday, September 09, 2010 8:25 PM > To: user@couchdb.apache.org > Subject: Couch not releasing deleted files > > [CouchDB Version : 0.11] > [OS: RHEL 5.5] > > Hi all, > > We are seeing some odd behaviour with CouchDb not releasing files after it > has compacted them meaning we are seeing disk space continue to increase > until the service is restarted even though we have regular compaction / > cleanup. > > Typical scenario : File system alert for disk space .. > > df says 5.5.GB used .. > > df -h | grep -P 'Filesystem|db' > > Filesystem Size Used Avail Use% Mounted on > > 8.0G 5.5G 2.2G 72% /var/lib/db > > > .. but the files on the disk (and in Futon) say only 793MB is being used .. > > > du -sh /var/lib/db/* > > 793M /var/lib/db/couchdb > > > .. running lsof shows the files marked as deleted but still there (I'm > assuming beam still has some kind of handle still open to them) .. > > > lsof | grep -P 'COMMAND|/var/lib/db/couchdb/' > > COMMAND PID USER FD TYPE DEVICE SIZE > NODE NAME > > beam.smp 21352 couchdb 12u REG 253,10 4183 > 1210374 > > /var/lib/db/couchdb/.active_context_design/8efe9584a944f4b24b97f134f541d80f.view > > beam.smp 21352 couchdb 16u REG 253,10 612039080 > 1204227 /var/lib/db/couchdb/active_context.couch (deleted) > > beam.smp 21352 couchdb 20u REG 253,10 180699242 > 1204237 /var/lib/db/couchdb/active_context.couch (deleted) > > beam.smp 21352 couchdb 21u REG 253,10 3133540 > 2023429 /var/lib/db/couchdb/thesaurus.couch (deleted) > > beam.smp 21352 couchdb 25u REG 253,10 71422052 > 2023428 /var/lib/db/couchdb/active_context_schedule.couch (deleted) > > beam.smp 21352 couchdb 26u REG 253,10 71372900 > 2056193 /var/lib/db/couchdb/active_context_schedule.couch > > beam.smp 21352 couchdb 28u REG 253,10 621252717 > 1204238 /var/lib/db/couchdb/active_context.couch > > beam.smp 21352 couchdb 29u REG 253,10 1740903 > 90113 /var/lib/db/couchdb/mapping_cache.couch > > beam.smp 21352 couchdb 30u REG 253,10 3133540 > 1204253 /var/lib/db/couchdb/thesaurus.couch (deleted) > > beam.smp 21352 couchdb 31u REG 253,10 2318439 > 1204240 /var/lib/db/couchdb/mappings.couch (deleted) > > beam.smp 21352 couchdb 32u REG 253,10 983150 > 2004994 > /var/lib/db/couchdb/.mappings_design/f9964c50b95c167c8ac3d8d3b4c76a4b.view > > beam.smp 21352 couchdb 33u REG 253,10 3133540 > 2056194 /var/lib/db/couchdb/thesaurus.couch > > beam.smp 21352 couchdb 34u REG 253,10 70742116 > 1204231 /var/lib/db/couchdb/active_context_schedule.couch (deleted) > > beam.smp 21352 couchdb 44u REG 253,10 1740903 > 2023427 /var/lib/db/couchdb/mapping_cache.couch (deleted) > > beam.smp 21352 couchdb 45u REG 253,10 65396836 > 1994756 /var/lib/db/couchdb/active_context_schedule.couch (deleted) > > beam.smp 21352 couchdb 47u REG 253,10 71229540 > 1204243 /var/lib/db/couchdb/active_context_schedule.couch (deleted) > > beam.smp 21352 couchdb 48u REG 253,10 240254078 > 1204226 /var/lib/db/couchdb/active_context.couch (deleted) > > beam.smp 21352 couchdb 49u REG 253,10 2232423 > 1204241 /var/lib/db/couchdb/mappings.couch > > beam.smp 21352 couchdb 51u REG 253,10 96903353 > 1210371 > > /var/lib/db/couchdb/.active_context_design/b406f7bb6d01b6d50324f14cde588324.view > > beam.smp 21352 couchdb 52u REG 253,10 1192034 > 1249282 > > /var/lib/db/couchdb/.mapping_cache_design/8bbf80a6e1ca8f2c71ebc90c730e7190.view > > beam.smp 21352 couchdb 53u REG 253,10 1622135 > 2004993 > /var/lib/db/couchdb/.mappings_design/1ff69004b575837aae9d4147162197d4.view > > beam.smp 21352 couchdb 54u REG 253,10 4172 > 1208327 > /var/lib/db/couchdb/.thesaurus_design/73fe79b20ff06e341257fa78231ca87e.view > > beam.smp 21352 couchdb 55u REG 253,10 13496483 > 1208325 > /var/lib/db/couchdb/.thesaurus_design/fcc08d6e41c54c0d470c5984ea9eb6e3.view > > beam.smp 21352 couchdb 56u REG 253,10 12959902 > 1208326 > /var/lib/db/couchdb/.thesaurus_design/c06195da19f34ced37fa1ab91e663c73.view > > beam.smp 21352 couchdb 57u REG 253,10 70254692 > 1204236 /var/lib/db/couchdb/active_context_schedule.couch (deleted) > > beam.smp 21352 couchdb 58u REG 253,10 581730 > 1210373 > > /var/lib/db/couchdb/.active_context_schedule_design/c64c047dcff74ea2122356efb1042759.view > > beam.smp 21352 couchdb 67u REG 253,10 1818727 > 1204247 /var/lib/db/couchdb/mapping_cache.couch (deleted) > > beam.smp 21352 couchdb 78u REG 253,10 609905146 > 1204234 /var/lib/db/couchdb/active_context.couch (deleted) > > beam.smp 21352 couchdb 80u REG 253,10 545919085 > 1994753 /var/lib/db/couchdb/active_context.couch (deleted) > > beam.smp 21352 couchdb 81u REG 253,10 71032932 > 1204249 /var/lib/db/couchdb/active_context_schedule.couch (deleted) > > beam.smp 21352 couchdb 84u REG 253,10 548866111 > 1204239 /var/lib/db/couchdb/active_context.couch (deleted) > > beam.smp 21352 couchdb 85u REG 253,10 564612402 > 1204233 /var/lib/db/couchdb/active_context.couch (deleted) > > beam.smp 21352 couchdb 87u REG 253,10 615996179 > 1204228 /var/lib/db/couchdb/active_context.couch (deleted) > > beam.smp 21352 couchdb 102u REG 253,10 561111149 > 2023425 /var/lib/db/couchdb/active_context.couch (deleted) > > > Has anyone seen this behaviour before or knows of a way around this ? > Currently we have to restart the couchdb service in order to restore the > disk space. > > Any help/advice is much appreciated. > > Regards > > Stephen > --001636c92cd2ad38bb048fe600fa--