couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: Multiple couchjs processes being spawned
Date Sun, 02 Jan 2011 22:10:48 GMT
Mahesh, do you by chance have an updater running for the view group when the view compaction
completes?

On Jan 2, 2011, at 4:53 PM, Adam Kocoloski wrote:

> These reports do sound suspiciously like the problems described in COUCHDB-901 that caused
us to rewrite the OS process management in BigCouch.  Mahesh, yours is the first report I've
heard that specifically implicated view compaction.  That's significant.  I didn't understand
what you meant by "clobber all the views", though.
> 
> I'll poke around and see if I can devise a scenario where the OS process would not be
released after view compaction.  In my experience the internal ets tables maintained in CouchDB
to track OS processes got out-of-sync - one table would report 1000 available processes, while
another table would claim only 2.  Best,
> 
> Adam
> 
> On Jan 2, 2011, at 12:25 PM, Mahesh Paolini-Subramanya wrote:
> 
>> Sorry - apologies - hit Send by accident :-(
>> 
>> Wow - wierdness here.  I'm running 1.0.1, and have noticed the same thing
>> happening.  The specific (and very replicable - for me at least) scenario is
>> as follows
>> - I've got around a thousand dbs, each with around 50K documents, and each
>> with about 5 (javascript) design documents.
>> - I clobber all the views, and remotely (LWP::UserAgent) compact the view
>> ($db/_compact/$design)
>> - THe compaction of the design document results in the design exactly being
>> generate (this is good_
>> - The compaction also leave a couchjs process just lying around (this would
>> be bad)
>> - After a few thousand couchjs processes, couchdb just chokes.
>> 
>> Any ideas? This related in any way to the process issues in BigCouch (Jira
>> -901)?
>> 
>> cheers
>> 
>> 
>> 
>> 
>> On Mon, Sep 7, 2009 at 12:18 AM, Paul Davis <paul.joseph.davis@gmail.com>wrote:
>> 
>>> Another random thought, after your clients have been running a bit, an
>>> easy way to check why type of stuff is going on with sockets is:
>>> 
>>> $ netstat -ap tcp
>>> 
>>> If your apps have periods of high turnover, check the sockets to see
>>> if you have lots of them in TIME_WAIT state.
>>> 
>>> Paul
>>> 
>>> On Sun, Sep 6, 2009 at 11:39 PM, Arun Thampi<arun.thampi@gmail.com> wrote:
>>>> Thanks Paul for the quick response. I'll poke around further to see if
>>> there
>>>> is anything amiss and will keep the list posted.
>>>> Cheers,
>>>> Arun
>>>> 
>>>> On Mon, Sep 7, 2009 at 11:24 AM, Paul Davis <paul.joseph.davis@gmail.com
>>>> wrote:
>>>> 
>>>>> The algorithm for CouchDB's use of OS processes is pretty simple:
>>>>> 
>>>>> "Give me an os process plz"
>>>>> 
>>>>> If none are available, then it creates a new one. A large number of OS
>>>>> processes would suggest to me that something isn't releasing OS
>>>>> processes correctly. My first guess would be to look at _list and
>>>>> _show as those both require an OS process, but are also at the mercy
>>>>> of client connection semantics. I might be reaching a bit, but I
>>>>> wonder if CouchRest is failing to close sockets properly. Its not much
>>>>> more than a random finger pointing, but there have been other errors
>>>>> recently that also suggest CouchRest isn't doing proper socket
>>>>> handling.
>>>>> 
>>>>> The other possibility is there's something weird with killing
>>>>> processes, but couchspawnkillable should've fixed that.
>>>>> 
>>>>> HTH,
>>>>> Paul Davis
>>>>> 
>>>>> On Sun, Sep 6, 2009 at 11:18 PM, Arun Thampi<arun.thampi@gmail.com>
>>> wrote:
>>>>>> Hi guys - Been running CouchDB trunk(r804727) in production for about
>>> 3
>>>>>> weeks now and one thing I've noticed is that the number of couchjs
>>>>> processes
>>>>>> (/usr/local/lib/couchdb/bin/couchjs
>>>>> /usr/local/share/couchdb/server/main.js)
>>>>>> keeps increasing to a large amount. Is this normal? Does CouchDB
>>> manage
>>>>>> these processes and eventually kill inactive couchjs processes?
>>>>>> Just FYI I'm using CouchRest as part of a Rails app to query two
>>>>> different
>>>>>> views in my db.
>>>>>> 
>>>>>> Thanks in advance.
>>>>>> 
>>>>>> Cheers,
>>>>>> Arun
>>>>>> 
>>>>>> --
>>>>>> It's better to be a pirate than join the Navy - Steve Jobs
>>>>>> http://mclov.in
>>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> --
>>>> It's better to be a pirate than join the Navy - Steve Jobs
>>>> http://mclov.in
>>>> 
>>> 
>> 
>> 
>> 
>> -- 
>> Mahesh Paolini-Subramanya | CTO | mahesh@aptela.com | 703.386.1500 Ext. 9100
>> 13454 Sunrise Valley Drive | Suite 500 | Herndon, VA
> 


Mime
View raw message