couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Randall Leeds (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-1282) Allow special value 0 for os_process_limit
Date Thu, 15 Sep 2011 01:48:08 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13105059#comment-13105059
] 

Randall Leeds commented on COUCHDB-1282:
----------------------------------------

Background:
This work stemmed from some experiments I conducted with refactoring the bulk update/validation
code in couch_db to validate in parallel. I found that when I did not restrict how many os
processes it used, and therefore used the full 25 default, performance was slightly worse
than when using only 2 or 4. However, almost all of this performance can be won by breaking
the request at the client into pieces and performing parallel bulk updates there. Adam also
correctly points out that view indexing ties up os processes for the entire duration of a
view index update and restricted this to such small numbers could potentially lead to all
the query server processes getting tied up.

Closing this for now. Ping me if you want to pick up a conversation about query server performance.

> Allow special value 0 for os_process_limit
> ------------------------------------------
>
>                 Key: COUCHDB-1282
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1282
>             Project: CouchDB
>          Issue Type: Improvement
>            Reporter: Randall Leeds
>            Assignee: Randall Leeds
>            Priority: Trivial
>         Attachments: 0001-Smart-os_proc_limit-default-COUCHDB-1282.patch
>
>
> It's unlikely to be beneficial to run more os processes than there are logical processors
in a system. It creates extra scheduling overhead if they are all in use. Currently default.ini
specifies 25, though the code specifies 10 if the option is missing, and I expect both are
too high for many real world deployments.
> I propose the value 0 be used in default.ini and that it take special meaning: use the
number of logical processors erlang detects.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message