couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Filipe Manana (Created) (JIRA)" <j...@apache.org>
Subject [jira] [Created] (COUCHDB-1375) couch_query_servers pool deadlock when running "os_process_limit" indexers
Date Thu, 05 Jan 2012 18:16:39 GMT
couch_query_servers pool deadlock when running "os_process_limit" indexers
--------------------------------------------------------------------------

                 Key: COUCHDB-1375
                 URL: https://issues.apache.org/jira/browse/COUCHDB-1375
             Project: CouchDB
          Issue Type: Bug
          Components: View Server Support
    Affects Versions: 1.1
            Reporter: Filipe Manana


When using external view servers, such as couchjs for example, if we trigger "os_process_limit"
(or more) index updates simultaneously, we can reach a deadlock case.

The issue is that each index updater will acquire a couchjs (os_proc record) process from
couch_query_servers to apply the map function against each document. After the indexer finishes,
it will release (return to couch_query_servers) the couchjs process used for the maps.

However, while the indexer is writing to the btrees, if the reduce function is a JavaScript
function (or any other language other than Erlang or a builtin reduce function), the function
is called often to reduce key-values - this results in the view updater process to ask couch_query_servers
for a another couchjs process (this is done on every reduce function call) - however "os_process_limit"
couchjs processes are already allocated to "os_process_limit" indexers for the mapping.

The solution here would be to have the index updater to preallocate a couchjs process for
the reduces and release it when it finishes (like it's done for the maps).

This only happens if the number of changes to index is greater than 500 (the work queue sizes).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message