couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Dionne <dio...@dionne-associates.com>
Subject Re: CouchDb not releasing files
Date Sat, 25 Sep 2010 15:07:27 GMT
couch_file has a close function which presumably does the right thing, but it's only called
from couch_db_update:terminate 



On Sep 25, 2010, at 11:02 AM, Randall Leeds wrote:

> What about if an exit() is called or something? I think then terminate
> is still called, but worth checking.
> 
> On Sat, Sep 25, 2010 at 16:57, Robert Dionne
> <dionne@dionne-associates.com> wrote:
>> http://erldocs.com/otp_src_R13B/stdlib/gen_server.html
>> 
>> If the function returns {stop,Reason,Reply,NewState}, Reply will be given back to
From. If the function returns {stop,Reason,NewState}, any reply to From must be given explicitly
using gen_server:reply/2. The gen_server will then call Module:terminate(Reason,NewState)and
terminate.
>> 
>> 
>> 
>> On Sep 25, 2010, at 10:53 AM, Paul Davis wrote:
>> 
>>> I'm not aware of a case where a gen_server will stop without calling
>>> terminate except if its a hard VM shutdown. Though, I get the feeling
>>> that Kocoloski is going to remind me of a case I've seen before and
>>> I'll say "Oh duh!" and then move on with my life.
>>> 
>>> HTH,
>>> Paul Davis
>>> 
>>> On Sat, Sep 25, 2010 at 8:35 AM, Robert Dionne
>>> <dionne@dionne-associates.com> wrote:
>>>> Filipe,
>>>> 
>>>>  Won't terminate be called only if the gen_server is stopped for a reason?
>>>> 
>>>> Bob
>>>> 
>>>> 
>>>> 
>>>> On Sep 25, 2010, at 7:30 AM, Filipe David Manana wrote:
>>>> 
>>>>> Stephen,
>>>>> 
>>>>> I committed something to trunk (
>>>>> http://svn.apache.org/viewvc?view=revision&revision=1001196 ) that
>>>>> might be the cause for your issue.
>>>>> Can you test it with trunk?
>>>>> 
>>>>> I was not yet able to reproduce the issue.
>>>>> 
>>>>> cheers
>>>>> 
>>>>> On Wed, Sep 22, 2010 at 2:40 PM, [mRg] <emargee@gmail.com> wrote:
>>>>>> Yes I did, issued it a couple of times as I was trying to see if
its was
>>>>>> some kind of race condition.
>>>>>> 
>>>>>> Have a site due to launch on Friday so have got crazy busy but ill
post more
>>>>>> info when I have it, unless anyone else can try and help replicate
this ?
>>>>>> 
>>>>>> On 20 September 2010 21:42, Paul Davis <paul.joseph.davis@gmail.com>
wrote:
>>>>>> 
>>>>>>> Do you issue two compact commands simultaneously or wait for
the first
>>>>>>> one to complete and then run the second?
>>>>>>> 
>>>>>>> 
>>>>>>> On Mon, Sep 20, 2010 at 4:07 PM, [mRg] <emargee@gmail.com>
wrote:
>>>>>>>> Well I can certainly reproduce the issue, but am having trouble
finding
>>>>>>> the
>>>>>>>> exact sequence (annoyingly of course)
>>>>>>>> 
>>>>>>>> 1> I started with a blank VM of Ubuntu 10.10 (ext3) running
on virtualbox
>>>>>>>> with latest CouchDB (apt-get install couchdb)
>>>>>>>> 2> Began adding lots of blank docs ..  (curl -X POST -H
>>>>>>>> "Content-Type:application/json" http://localhost:5984/test
-d {})
>>>>>>>> 3> Created a simple view ("function(doc) {  emit(null,
doc); }") and ran
>>>>>>> to
>>>>>>>> ensure it wrote to disk.
>>>>>>>> 4> I issued 2 compact commands (curl -X POST
>>>>>>>> http://localhost:5984/test/_compact/ -H "Content-Type:application/json")
>>>>>>>> 5> And began adding more documents again as before ..
>>>>>>>> 
>>>>>>>> Repeating this, essentially creating view data and compacting
it about 10
>>>>>>>> times resulted in 4 'snagged' files
>>>>>>>> 
>>>>>>>> root@ubuntu:~# lsof | grep -P 'COMMAND|/var/lib/couchdb/1.0.1'
>>>>>>>> COMMAND    PID    USER   FD      TYPE     DEVICE SIZE/OFF
      NODE NAME
>>>>>>>> beam      1083 couchdb   13u      REG      252,0     4185
       271
>>>>>>>> /var/lib/couchdb/1.0.1/_users.couch
>>>>>>>>>> beam      1083 couchdb   14u      REG      252,0
  331865        275
>>>>>>>> /var/lib/couchdb/1.0.1/.delete/4e000e0ae5cce3d275f09b4542396e85
(deleted)
>>>>>>>>>> beam      1083 couchdb   16u      REG      252,0
  602210        279
>>>>>>>> /var/lib/couchdb/1.0.1/.delete/a9ee2b96c12591455ea795fa5324dc9c
(deleted)
>>>>>>>> beam      1083 couchdb   17u      REG      252,0   270434
       283
>>>>>>>> /var/lib/couchdb/1.0.1/.test_design/07ca32cf9b0de9c915c5d9ce653cdca3.view
>>>>>>>> beam      1083 couchdb   18u      REG      252,0   204898
       284
>>>>>>>> /var/lib/couchdb/1.0.1/.test_design/540958c4124af3925fe467afb96f4906.view
>>>>>>>> beam      1083 couchdb   20u      REG      252,0   405602
       286
>>>>>>>> /var/lib/couchdb/1.0.1/.test_design/f26a8fcc3d2ce226a9e652338882c3db.view
>>>>>>>> beam      1083 couchdb   21u      REG      252,0   299106
       287
>>>>>>>> /var/lib/couchdb/1.0.1/.test_design/40614f8c8e1b4bab9d093881e914729d.view
>>>>>>>>>> beam      1083 couchdb   22u      REG      252,0
  106594        285
>>>>>>>> /var/lib/couchdb/1.0.1/.delete/31909e936ce94db7a6cede72827b18b2
(deleted)
>>>>>>>>>> beam      1083 couchdb   24r      REG      252,0
  233570        288
>>>>>>>> /var/lib/couchdb/1.0.1/.delete/83d09ec6dab90ca6078c0310085b97cc
(deleted)
>>>>>>>> beam      1083 couchdb   26u      REG      252,0   163932
       281
>>>>>>>> /var/lib/couchdb/1.0.1/.test_design/3486e8de398e27b8767afd4975691360.view
>>>>>>>> beam      1083 couchdb   27w      REG      252,0   114786
       289
>>>>>>>> /var/lib/couchdb/1.0.1/test.couch
>>>>>>>> beam      1083 couchdb   28u      REG      252,0   217186
       280
>>>>>>>> /var/lib/couchdb/1.0.1/.test_design/b450740e3245f89ab902db10d767a397.view
>>>>>>>> 
>>>>>>>> .. while the bottom 2 marked with (deleted) hung around for
about 20
>>>>>>> minutes
>>>>>>>> they eventually disappeared ..
>>>>>>>> 
>>>>>>>> root@ubuntu:~# lsof | grep -P 'COMMAND|/var/lib/couchdb/1.0.1'
>>>>>>>> COMMAND    PID    USER   FD      TYPE     DEVICE SIZE/OFF
      NODE NAME
>>>>>>>> beam      1083 couchdb   13u      REG      252,0     4185
       271
>>>>>>>> /var/lib/couchdb/1.0.1/_users.couch
>>>>>>>>>> beam      1083 couchdb   14u      REG      252,0
  331865        275
>>>>>>>> /var/lib/couchdb/1.0.1/.delete/4e000e0ae5cce3d275f09b4542396e85
(deleted)
>>>>>>>>>> beam      1083 couchdb   16u      REG      252,0
  602210        279
>>>>>>>> /var/lib/couchdb/1.0.1/.delete/a9ee2b96c12591455ea795fa5324dc9c
(deleted)
>>>>>>>> beam      1083 couchdb   17u      REG      252,0   270434
       283
>>>>>>>> /var/lib/couchdb/1.0.1/.test_design/07ca32cf9b0de9c915c5d9ce653cdca3.view
>>>>>>>> beam      1083 couchdb   18u      REG      252,0   204898
       284
>>>>>>>> /var/lib/couchdb/1.0.1/.test_design/540958c4124af3925fe467afb96f4906.view
>>>>>>>> beam      1083 couchdb   20u      REG      252,0   405602
       286
>>>>>>>> /var/lib/couchdb/1.0.1/.test_design/f26a8fcc3d2ce226a9e652338882c3db.view
>>>>>>>> beam      1083 couchdb   21u      REG      252,0   299106
       287
>>>>>>>> /var/lib/couchdb/1.0.1/.test_design/40614f8c8e1b4bab9d093881e914729d.view
>>>>>>>> beam      1083 couchdb   23u      REG      252,0   114786
       288
>>>>>>>> /var/lib/couchdb/1.0.1/test.couch
>>>>>>>> beam      1083 couchdb   26u      REG      252,0   163932
       281
>>>>>>>> /var/lib/couchdb/1.0.1/.test_design/3486e8de398e27b8767afd4975691360.view
>>>>>>>> beam      1083 couchdb   28u      REG      252,0   217186
       280
>>>>>>>> /var/lib/couchdb/1.0.1/.test_design/b450740e3245f89ab902db10d767a397.view
>>>>>>>> 
>>>>>>>> .. but the top 2 did not and are still hanging around over
an hour later
>>>>>>> ..
>>>>>>>> 
>>>>>>>> I will keep looking to see if I can find the exact sequence
..
>>>>>>>> 
>>>>>>>> On 20 September 2010 19:09, [mRg] <emargee@gmail.com>
wrote:
>>>>>>>> 
>>>>>>>>> I am trying to reproduce it currently on a fresh VM.
It seems to be a
>>>>>>>>> symptom of view compaction, after the file is moved to
the .delete
>>>>>>> directory
>>>>>>>>> and then finally deleted is where the problem seems to
occur.
>>>>>>>>> 
>>>>>>>>> I tried with a fresh ubuntu 10.10 but I couldnt reproduce
but the system
>>>>>>>>> I'm seeing it on is ext3 (ubuntu was ext4) so I'm going
to start again
>>>>>>> with
>>>>>>>>> that and report back.
>>>>>>>>> 
>>>>>>>>> I did wonder how this would be reproduced in a test as
you have to get
>>>>>>>>> quite low level (using lsof) to see if the issue is manifesting
itself
>>>>>>> :)
>>>>>>>>> 
>>>>>>>>> Regards
>>>>>>>>> 
>>>>>>>>> Stephen
>>>>>>>>> 
>>>>>>>>> On 20 September 2010 16:00, Chris Anderson <jchris@apache.org>
wrote:
>>>>>>>>> 
>>>>>>>>>> On Sun, Sep 19, 2010 at 3:21 PM, Randall Leeds <
>>>>>>> randall.leeds@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>> If the bug is confirmed it should be on JIRA
if it is not already. If
>>>>>>>>>>> you have a test case that reproduces it that
would be fanstastic
>>>>>>>>>>> (bonus points for a JS test in Futon).
>>>>>>>>>>> 
>>>>>>>>>>> It's my opinion something this serious should
block 1.1, but
>>>>>>>>>>> ultimately that is up to the committers to determine,
yes?
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> I think this is pretty serious. It's probably a simple
fix, so as soon
>>>>>>>>>> as we have it reproducible, I'm sure it will get
fixed.
>>>>>>>>>> 
>>>>>>>>>> I'm not sure how it can be tested from JavaScript,
any ideas?
>>>>>>>>>> 
>>>>>>>>>> If not it should be testable via the Erlang tests.
>>>>>>>>>> 
>>>>>>>>>> Chris
>>>>>>>>>> 
>>>>>>>>>>> On Sun, Sep 19, 2010 at 22:09, [mRg] <emargee@gmail.com>
wrote:
>>>>>>>>>>>> Hi all,
>>>>>>>>>>>> 
>>>>>>>>>>>> This is just a cross-post to highlight a
thread on the user list.
>>>>>>> (with
>>>>>>>>>> the
>>>>>>>>>>>> same name as this one - all details etc are
on there, happy to
>>>>>>> repeat
>>>>>>>>>> here
>>>>>>>>>>>> if needed). It seems this problem was discussed
by some devs at
>>>>>>>>>> CouchCamp
>>>>>>>>>>>> and seems others are suffering this issue.
I was just wondering if
>>>>>>>>>> there was
>>>>>>>>>>>> a JIRA issue related to this that I/we can
track and if a fix for
>>>>>>> this
>>>>>>>>>> will
>>>>>>>>>>>> be included in any upcoming released (1.1
?).
>>>>>>>>>>>> 
>>>>>>>>>>>> Regards
>>>>>>>>>>>> 
>>>>>>>>>>>> Stephen
>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> --
>>>>>>>>>> Chris Anderson
>>>>>>>>>> http://jchrisa.net
>>>>>>>>>> http://couch.io
>>>>>>>>>> 
>>>>>>>>> 
>>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> Filipe David Manana,
>>>>> fdmanana@gmail.com, fdmanana@apache.org
>>>>> 
>>>>> "Reasonable men adapt themselves to the world.
>>>>>  Unreasonable men adapt the world to themselves.
>>>>>  That's why all progress depends on unreasonable men."
>>>> 
>>>> 
>> 
>> 


Mime
View raw message