Mailing-List: contact dev-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of emargee@gmail.com designates
 209.85.212.180 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type;
        b=x1iJC49k3I/gm7K3oJyOzaEpGc0M9IxnNi0L1BzZ+kqcE012nPLFgh54onHooxnu2j
         i5qh+Gwib3ufjRvXLEv2g00A/Lilv5EXISOhg02pLjSXnHowmS6K47cO6j4t9LviN3XC
         K3ZbVbWq7+6k1zU9X9ZNNUlKOa2s/dm3SltlI=
MIME-Version: 1.0
In-Reply-To: <AANLkTinchBr=REHg2t=ce2w4nhvb1JvogHJgSV1rTfO3@mail.gmail.com>
References: <AANLkTinmRGyCjVoKEceV5w6eYJdPDAW9WJY9MvRvWM57@mail.gmail.com>
	<AANLkTimbFKGnbX-WtFn7B=TG7Gkhq1evfEiuRD4e6=U-@mail.gmail.com>
	<AANLkTimKfUG273G8etu40O6wOAwbzsY3rvPN77Ju8B0P@mail.gmail.com>
	<AANLkTikzt32LCjHuKSH0CS18h4U9ZXg=apyBJ+=oZYFL@mail.gmail.com>
	<AANLkTi=6vCEqS5enHQx=-U_T0koWNxfaVkg3d9_tbaxi@mail.gmail.com>
	<AANLkTinchBr=REHg2t=ce2w4nhvb1JvogHJgSV1rTfO3@mail.gmail.com>
Date: Wed, 22 Sep 2010 14:40:03 +0100
Message-ID: <AANLkTi=CXxmYQqp1tCgPAkU0XUEZFAt6_3spKzCPEBFp@mail.gmail.com>
Subject: Re: CouchDb not releasing files
From: "[mRg]" <emargee@gmail.com>
To: dev@couchdb.apache.org
Content-Type: multipart/alternative; boundary=0016364583aa6223ae0490d948a2

--0016364583aa6223ae0490d948a2
Content-Type: text/plain; charset=ISO-8859-1

Yes I did, issued it a couple of times as I was trying to see if its was
some kind of race condition.

Have a site due to launch on Friday so have got crazy busy but ill post more
info when I have it, unless anyone else can try and help replicate this ?

On 20 September 2010 21:42, Paul Davis <paul.joseph.davis@gmail.com> wrote:

> Do you issue two compact commands simultaneously or wait for the first
> one to complete and then run the second?
>
>
> On Mon, Sep 20, 2010 at 4:07 PM, [mRg] <emargee@gmail.com> wrote:
> > Well I can certainly reproduce the issue, but am having trouble finding
> the
> > exact sequence (annoyingly of course)
> >
> > 1> I started with a blank VM of Ubuntu 10.10 (ext3) running on virtualbox
> > with latest CouchDB (apt-get install couchdb)
> > 2> Began adding lots of blank docs ..  (curl -X POST -H
> > "Content-Type:application/json" http://localhost:5984/test -d {})
> > 3> Created a simple view ("function(doc) {  emit(null, doc); }") and ran
> to
> > ensure it wrote to disk.
> > 4> I issued 2 compact commands (curl -X POST
> > http://localhost:5984/test/_compact/ -H "Content-Type:application/json")
> > 5> And began adding more documents again as before ..
> >
> > Repeating this, essentially creating view data and compacting it about 10
> > times resulted in 4 'snagged' files
> >
> > root@ubuntu:~# lsof | grep -P 'COMMAND|/var/lib/couchdb/1.0.1'
> > COMMAND    PID    USER   FD      TYPE     DEVICE SIZE/OFF       NODE NAME
> > beam      1083 couchdb   13u      REG      252,0     4185        271
> > /var/lib/couchdb/1.0.1/_users.couch
> >>> beam      1083 couchdb   14u      REG      252,0   331865        275
> > /var/lib/couchdb/1.0.1/.delete/4e000e0ae5cce3d275f09b4542396e85 (deleted)
> >>> beam      1083 couchdb   16u      REG      252,0   602210        279
> > /var/lib/couchdb/1.0.1/.delete/a9ee2b96c12591455ea795fa5324dc9c (deleted)
> > beam      1083 couchdb   17u      REG      252,0   270434        283
> > /var/lib/couchdb/1.0.1/.test_design/07ca32cf9b0de9c915c5d9ce653cdca3.view
> > beam      1083 couchdb   18u      REG      252,0   204898        284
> > /var/lib/couchdb/1.0.1/.test_design/540958c4124af3925fe467afb96f4906.view
> > beam      1083 couchdb   20u      REG      252,0   405602        286
> > /var/lib/couchdb/1.0.1/.test_design/f26a8fcc3d2ce226a9e652338882c3db.view
> > beam      1083 couchdb   21u      REG      252,0   299106        287
> > /var/lib/couchdb/1.0.1/.test_design/40614f8c8e1b4bab9d093881e914729d.view
> >>> beam      1083 couchdb   22u      REG      252,0   106594        285
> > /var/lib/couchdb/1.0.1/.delete/31909e936ce94db7a6cede72827b18b2 (deleted)
> >>> beam      1083 couchdb   24r      REG      252,0   233570        288
> > /var/lib/couchdb/1.0.1/.delete/83d09ec6dab90ca6078c0310085b97cc (deleted)
> > beam      1083 couchdb   26u      REG      252,0   163932        281
> > /var/lib/couchdb/1.0.1/.test_design/3486e8de398e27b8767afd4975691360.view
> > beam      1083 couchdb   27w      REG      252,0   114786        289
> > /var/lib/couchdb/1.0.1/test.couch
> > beam      1083 couchdb   28u      REG      252,0   217186        280
> > /var/lib/couchdb/1.0.1/.test_design/b450740e3245f89ab902db10d767a397.view
> >
> > .. while the bottom 2 marked with (deleted) hung around for about 20
> minutes
> > they eventually disappeared ..
> >
> > root@ubuntu:~# lsof | grep -P 'COMMAND|/var/lib/couchdb/1.0.1'
> > COMMAND    PID    USER   FD      TYPE     DEVICE SIZE/OFF       NODE NAME
> > beam      1083 couchdb   13u      REG      252,0     4185        271
> > /var/lib/couchdb/1.0.1/_users.couch
> >>> beam      1083 couchdb   14u      REG      252,0   331865        275
> > /var/lib/couchdb/1.0.1/.delete/4e000e0ae5cce3d275f09b4542396e85 (deleted)
> >>> beam      1083 couchdb   16u      REG      252,0   602210        279
> > /var/lib/couchdb/1.0.1/.delete/a9ee2b96c12591455ea795fa5324dc9c (deleted)
> > beam      1083 couchdb   17u      REG      252,0   270434        283
> > /var/lib/couchdb/1.0.1/.test_design/07ca32cf9b0de9c915c5d9ce653cdca3.view
> > beam      1083 couchdb   18u      REG      252,0   204898        284
> > /var/lib/couchdb/1.0.1/.test_design/540958c4124af3925fe467afb96f4906.view
> > beam      1083 couchdb   20u      REG      252,0   405602        286
> > /var/lib/couchdb/1.0.1/.test_design/f26a8fcc3d2ce226a9e652338882c3db.view
> > beam      1083 couchdb   21u      REG      252,0   299106        287
> > /var/lib/couchdb/1.0.1/.test_design/40614f8c8e1b4bab9d093881e914729d.view
> > beam      1083 couchdb   23u      REG      252,0   114786        288
> > /var/lib/couchdb/1.0.1/test.couch
> > beam      1083 couchdb   26u      REG      252,0   163932        281
> > /var/lib/couchdb/1.0.1/.test_design/3486e8de398e27b8767afd4975691360.view
> > beam      1083 couchdb   28u      REG      252,0   217186        280
> > /var/lib/couchdb/1.0.1/.test_design/b450740e3245f89ab902db10d767a397.view
> >
> > .. but the top 2 did not and are still hanging around over an hour later
> ..
> >
> > I will keep looking to see if I can find the exact sequence ..
> >
> > On 20 September 2010 19:09, [mRg] <emargee@gmail.com> wrote:
> >
> >> I am trying to reproduce it currently on a fresh VM. It seems to be a
> >> symptom of view compaction, after the file is moved to the .delete
> directory
> >> and then finally deleted is where the problem seems to occur.
> >>
> >> I tried with a fresh ubuntu 10.10 but I couldnt reproduce but the system
> >> I'm seeing it on is ext3 (ubuntu was ext4) so I'm going to start again
> with
> >> that and report back.
> >>
> >> I did wonder how this would be reproduced in a test as you have to get
> >> quite low level (using lsof) to see if the issue is manifesting itself
> :)
> >>
> >> Regards
> >>
> >> Stephen
> >>
> >> On 20 September 2010 16:00, Chris Anderson <jchris@apache.org> wrote:
> >>
> >>> On Sun, Sep 19, 2010 at 3:21 PM, Randall Leeds <
> randall.leeds@gmail.com>
> >>> wrote:
> >>> > If the bug is confirmed it should be on JIRA if it is not already. If
> >>> > you have a test case that reproduces it that would be fanstastic
> >>> > (bonus points for a JS test in Futon).
> >>> >
> >>> > It's my opinion something this serious should block 1.1, but
> >>> > ultimately that is up to the committers to determine, yes?
> >>> >
> >>>
> >>> I think this is pretty serious. It's probably a simple fix, so as soon
> >>> as we have it reproducible, I'm sure it will get fixed.
> >>>
> >>> I'm not sure how it can be tested from JavaScript, any ideas?
> >>>
> >>> If not it should be testable via the Erlang tests.
> >>>
> >>> Chris
> >>>
> >>> > On Sun, Sep 19, 2010 at 22:09, [mRg] <emargee@gmail.com> wrote:
> >>> >> Hi all,
> >>> >>
> >>> >> This is just a cross-post to highlight a thread on the user list.
> (with
> >>> the
> >>> >> same name as this one - all details etc are on there, happy to
> repeat
> >>> here
> >>> >> if needed). It seems this problem was discussed by some devs at
> >>> CouchCamp
> >>> >> and seems others are suffering this issue. I was just wondering if
> >>> there was
> >>> >> a JIRA issue related to this that I/we can track and if a fix for
> this
> >>> will
> >>> >> be included in any upcoming released (1.1 ?).
> >>> >>
> >>> >> Regards
> >>> >>
> >>> >> Stephen
> >>> >>
> >>> >
> >>>
> >>>
> >>>
> >>> --
> >>> Chris Anderson
> >>> http://jchrisa.net
> >>> http://couch.io
> >>>
> >>
> >>
> >
>

--0016364583aa6223ae0490d948a2--