couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "[mRg]" <emar...@gmail.com>
Subject Re: CouchDb not releasing files
Date Mon, 20 Sep 2010 20:07:20 GMT
Well I can certainly reproduce the issue, but am having trouble finding the
exact sequence (annoyingly of course)

1> I started with a blank VM of Ubuntu 10.10 (ext3) running on virtualbox
with latest CouchDB (apt-get install couchdb)
2> Began adding lots of blank docs ..  (curl -X POST -H
"Content-Type:application/json" http://localhost:5984/test -d {})
3> Created a simple view ("function(doc) {  emit(null, doc); }") and ran to
ensure it wrote to disk.
4> I issued 2 compact commands (curl -X POST
http://localhost:5984/test/_compact/ -H "Content-Type:application/json")
5> And began adding more documents again as before ..

Repeating this, essentially creating view data and compacting it about 10
times resulted in 4 'snagged' files

root@ubuntu:~# lsof | grep -P 'COMMAND|/var/lib/couchdb/1.0.1'
COMMAND    PID    USER   FD      TYPE     DEVICE SIZE/OFF       NODE NAME
beam      1083 couchdb   13u      REG      252,0     4185        271
/var/lib/couchdb/1.0.1/_users.couch
>> beam      1083 couchdb   14u      REG      252,0   331865        275
/var/lib/couchdb/1.0.1/.delete/4e000e0ae5cce3d275f09b4542396e85 (deleted)
>> beam      1083 couchdb   16u      REG      252,0   602210        279
/var/lib/couchdb/1.0.1/.delete/a9ee2b96c12591455ea795fa5324dc9c (deleted)
beam      1083 couchdb   17u      REG      252,0   270434        283
/var/lib/couchdb/1.0.1/.test_design/07ca32cf9b0de9c915c5d9ce653cdca3.view
beam      1083 couchdb   18u      REG      252,0   204898        284
/var/lib/couchdb/1.0.1/.test_design/540958c4124af3925fe467afb96f4906.view
beam      1083 couchdb   20u      REG      252,0   405602        286
/var/lib/couchdb/1.0.1/.test_design/f26a8fcc3d2ce226a9e652338882c3db.view
beam      1083 couchdb   21u      REG      252,0   299106        287
/var/lib/couchdb/1.0.1/.test_design/40614f8c8e1b4bab9d093881e914729d.view
>> beam      1083 couchdb   22u      REG      252,0   106594        285
/var/lib/couchdb/1.0.1/.delete/31909e936ce94db7a6cede72827b18b2 (deleted)
>> beam      1083 couchdb   24r      REG      252,0   233570        288
/var/lib/couchdb/1.0.1/.delete/83d09ec6dab90ca6078c0310085b97cc (deleted)
beam      1083 couchdb   26u      REG      252,0   163932        281
/var/lib/couchdb/1.0.1/.test_design/3486e8de398e27b8767afd4975691360.view
beam      1083 couchdb   27w      REG      252,0   114786        289
/var/lib/couchdb/1.0.1/test.couch
beam      1083 couchdb   28u      REG      252,0   217186        280
/var/lib/couchdb/1.0.1/.test_design/b450740e3245f89ab902db10d767a397.view

.. while the bottom 2 marked with (deleted) hung around for about 20 minutes
they eventually disappeared ..

root@ubuntu:~# lsof | grep -P 'COMMAND|/var/lib/couchdb/1.0.1'
COMMAND    PID    USER   FD      TYPE     DEVICE SIZE/OFF       NODE NAME
beam      1083 couchdb   13u      REG      252,0     4185        271
/var/lib/couchdb/1.0.1/_users.couch
>> beam      1083 couchdb   14u      REG      252,0   331865        275
/var/lib/couchdb/1.0.1/.delete/4e000e0ae5cce3d275f09b4542396e85 (deleted)
>> beam      1083 couchdb   16u      REG      252,0   602210        279
/var/lib/couchdb/1.0.1/.delete/a9ee2b96c12591455ea795fa5324dc9c (deleted)
beam      1083 couchdb   17u      REG      252,0   270434        283
/var/lib/couchdb/1.0.1/.test_design/07ca32cf9b0de9c915c5d9ce653cdca3.view
beam      1083 couchdb   18u      REG      252,0   204898        284
/var/lib/couchdb/1.0.1/.test_design/540958c4124af3925fe467afb96f4906.view
beam      1083 couchdb   20u      REG      252,0   405602        286
/var/lib/couchdb/1.0.1/.test_design/f26a8fcc3d2ce226a9e652338882c3db.view
beam      1083 couchdb   21u      REG      252,0   299106        287
/var/lib/couchdb/1.0.1/.test_design/40614f8c8e1b4bab9d093881e914729d.view
beam      1083 couchdb   23u      REG      252,0   114786        288
/var/lib/couchdb/1.0.1/test.couch
beam      1083 couchdb   26u      REG      252,0   163932        281
/var/lib/couchdb/1.0.1/.test_design/3486e8de398e27b8767afd4975691360.view
beam      1083 couchdb   28u      REG      252,0   217186        280
/var/lib/couchdb/1.0.1/.test_design/b450740e3245f89ab902db10d767a397.view

.. but the top 2 did not and are still hanging around over an hour later ..

I will keep looking to see if I can find the exact sequence ..

On 20 September 2010 19:09, [mRg] <emargee@gmail.com> wrote:

> I am trying to reproduce it currently on a fresh VM. It seems to be a
> symptom of view compaction, after the file is moved to the .delete directory
> and then finally deleted is where the problem seems to occur.
>
> I tried with a fresh ubuntu 10.10 but I couldnt reproduce but the system
> I'm seeing it on is ext3 (ubuntu was ext4) so I'm going to start again with
> that and report back.
>
> I did wonder how this would be reproduced in a test as you have to get
> quite low level (using lsof) to see if the issue is manifesting itself :)
>
> Regards
>
> Stephen
>
> On 20 September 2010 16:00, Chris Anderson <jchris@apache.org> wrote:
>
>> On Sun, Sep 19, 2010 at 3:21 PM, Randall Leeds <randall.leeds@gmail.com>
>> wrote:
>> > If the bug is confirmed it should be on JIRA if it is not already. If
>> > you have a test case that reproduces it that would be fanstastic
>> > (bonus points for a JS test in Futon).
>> >
>> > It's my opinion something this serious should block 1.1, but
>> > ultimately that is up to the committers to determine, yes?
>> >
>>
>> I think this is pretty serious. It's probably a simple fix, so as soon
>> as we have it reproducible, I'm sure it will get fixed.
>>
>> I'm not sure how it can be tested from JavaScript, any ideas?
>>
>> If not it should be testable via the Erlang tests.
>>
>> Chris
>>
>> > On Sun, Sep 19, 2010 at 22:09, [mRg] <emargee@gmail.com> wrote:
>> >> Hi all,
>> >>
>> >> This is just a cross-post to highlight a thread on the user list. (with
>> the
>> >> same name as this one - all details etc are on there, happy to repeat
>> here
>> >> if needed). It seems this problem was discussed by some devs at
>> CouchCamp
>> >> and seems others are suffering this issue. I was just wondering if
>> there was
>> >> a JIRA issue related to this that I/we can track and if a fix for this
>> will
>> >> be included in any upcoming released (1.1 ?).
>> >>
>> >> Regards
>> >>
>> >> Stephen
>> >>
>> >
>>
>>
>>
>> --
>> Chris Anderson
>> http://jchrisa.net
>> http://couch.io
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message