couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Samuel Williams (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-1743) Make the view server & protocol faster
Date Mon, 29 Apr 2013 15:30:16 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-1743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13644579#comment-13644579
] 

Samuel Williams commented on COUCHDB-1743:
------------------------------------------

Robert, I humbly assert that all view servers, as it currently stands, are stateful, despite
the goals of the existing design. In addition, many scripting languages, are stateful in many
ways, for example, in a Python context, importing a module modifies global state in the Python
interpreter.

Most modern operating systems are perfectly happy managing multiple processes. My proposal
suggests that we have a 1-to-1 mapping between a design document and a process. One proposed
extension, the parallel processing extension, allows us to correctly deal with multiple chronologically
independent tasks. It also improves memory management by reducing the time a process is kept
running. Despite best intentions, lots of scripting environments have memory management inefficiencies.

Finally, there are issues relating to the current design which are inherently broken. For
example, it is impossible to load libraries in the reduce process.

I suggest that new query server in Python, according to my basic specification would be at
most 200 lines of code, if not significantly less. The benefit would be long term maintainability
and extensibility. Also, because the RPC protocol I suggested is basically similar to existing
RPC mechanisms, the transport protocol could be changed in the future with minimal trouble,
e.g. compression, shared memory segment, binary JSON, etc. 

I guess its similar to the OSI networking model in some ways. Currently CouchDB is combining
application layer and transport into one protocol. I'm trying to separate it, so that chances
in application layer don't have to elicit changes in the transport, and vice versa.

                
> Make the view server & protocol faster
> --------------------------------------
>
>                 Key: COUCHDB-1743
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1743
>             Project: CouchDB
>          Issue Type: Improvement
>            Reporter: Dave Cottlehuber
>              Labels: couchdb, erlang, gsoc2013, html, javascript, nodejs, rest
>
> View server protocol enhancements/refactoring - unix sockets, pipelining, different wire
format etc. Faster!!

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message