couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahesh Paolini-Subramanya <mah...@aptela.com>
Subject Re: Multiple couchjs processes being spawned
Date Mon, 03 Jan 2011 00:03:38 GMT
Updater?  Mildly confused....  (sorry 'bout that - terminology incomprehension on my part...)

cheers
Mahesh Paolini-Subramanya | CTO | mahesh@aptela.com | 703.386.1500 Ext. 9100
2250 Corporate Park Drive | Suite 150 | Herndon, VA | www.aptela.com
Check out our Blog | Follow us on Twitter | Refer a Friend 



On Jan 2, 2011, at 5:10 PM, Adam Kocoloski wrote:

> Mahesh, do you by chance have an updater running for the view group when the view compaction
completes?
> 
> On Jan 2, 2011, at 4:53 PM, Adam Kocoloski wrote:
> 
>> These reports do sound suspiciously like the problems described in COUCHDB-901 that
caused us to rewrite the OS process management in BigCouch.  Mahesh, yours is the first report
I've heard that specifically implicated view compaction.  That's significant.  I didn't understand
what you meant by "clobber all the views", though.
>> 
>> I'll poke around and see if I can devise a scenario where the OS process would not
be released after view compaction.  In my experience the internal ets tables maintained in
CouchDB to track OS processes got out-of-sync - one table would report 1000 available processes,
while another table would claim only 2.  Best,
>> 
>> Adam
>> 
>> On Jan 2, 2011, at 12:25 PM, Mahesh Paolini-Subramanya wrote:
>> 
>>> Sorry - apologies - hit Send by accident :-(
>>> 
>>> Wow - wierdness here.  I'm running 1.0.1, and have noticed the same thing
>>> happening.  The specific (and very replicable - for me at least) scenario is
>>> as follows
>>> - I've got around a thousand dbs, each with around 50K documents, and each
>>> with about 5 (javascript) design documents.
>>> - I clobber all the views, and remotely (LWP::UserAgent) compact the view
>>> ($db/_compact/$design)
>>> - THe compaction of the design document results in the design exactly being
>>> generate (this is good_
>>> - The compaction also leave a couchjs process just lying around (this would
>>> be bad)
>>> - After a few thousand couchjs processes, couchdb just chokes.
>>> 
>>> Any ideas? This related in any way to the process issues in BigCouch (Jira
>>> -901)?
>>> 
>>> cheers
>>> 
>>> 
>>> 
>>> 
>>> On Mon, Sep 7, 2009 at 12:18 AM, Paul Davis <paul.joseph.davis@gmail.com>wrote:
>>> 
>>>> Another random thought, after your clients have been running a bit, an
>>>> easy way to check why type of stuff is going on with sockets is:
>>>> 
>>>> $ netstat -ap tcp
>>>> 
>>>> If your apps have periods of high turnover, check the sockets to see
>>>> if you have lots of them in TIME_WAIT state.
>>>> 
>>>> Paul
>>>> 
>>>> On Sun, Sep 6, 2009 at 11:39 PM, Arun Thampi<arun.thampi@gmail.com>
wrote:
>>>>> Thanks Paul for the quick response. I'll poke around further to see if
>>>> there
>>>>> is anything amiss and will keep the list posted.
>>>>> Cheers,
>>>>> Arun
>>>>> 
>>>>> On Mon, Sep 7, 2009 at 11:24 AM, Paul Davis <paul.joseph.davis@gmail.com
>>>>> wrote:
>>>>> 
>>>>>> The algorithm for CouchDB's use of OS processes is pretty simple:
>>>>>> 
>>>>>> "Give me an os process plz"
>>>>>> 
>>>>>> If none are available, then it creates a new one. A large number
of OS
>>>>>> processes would suggest to me that something isn't releasing OS
>>>>>> processes correctly. My first guess would be to look at _list and
>>>>>> _show as those both require an OS process, but are also at the mercy
>>>>>> of client connection semantics. I might be reaching a bit, but I
>>>>>> wonder if CouchRest is failing to close sockets properly. Its not
much
>>>>>> more than a random finger pointing, but there have been other errors
>>>>>> recently that also suggest CouchRest isn't doing proper socket
>>>>>> handling.
>>>>>> 
>>>>>> The other possibility is there's something weird with killing
>>>>>> processes, but couchspawnkillable should've fixed that.
>>>>>> 
>>>>>> HTH,
>>>>>> Paul Davis
>>>>>> 
>>>>>> On Sun, Sep 6, 2009 at 11:18 PM, Arun Thampi<arun.thampi@gmail.com>
>>>> wrote:
>>>>>>> Hi guys - Been running CouchDB trunk(r804727) in production for
about
>>>> 3
>>>>>>> weeks now and one thing I've noticed is that the number of couchjs
>>>>>> processes
>>>>>>> (/usr/local/lib/couchdb/bin/couchjs
>>>>>> /usr/local/share/couchdb/server/main.js)
>>>>>>> keeps increasing to a large amount. Is this normal? Does CouchDB
>>>> manage
>>>>>>> these processes and eventually kill inactive couchjs processes?
>>>>>>> Just FYI I'm using CouchRest as part of a Rails app to query
two
>>>>>> different
>>>>>>> views in my db.
>>>>>>> 
>>>>>>> Thanks in advance.
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Arun
>>>>>>> 
>>>>>>> --
>>>>>>> It's better to be a pirate than join the Navy - Steve Jobs
>>>>>>> http://mclov.in
>>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> --
>>>>> It's better to be a pirate than join the Navy - Steve Jobs
>>>>> http://mclov.in
>>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> -- 
>>> Mahesh Paolini-Subramanya | CTO | mahesh@aptela.com | 703.386.1500 Ext. 9100
>>> 13454 Sunrise Valley Drive | Suite 500 | Herndon, VA
>> 
> 


Mime
View raw message