couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Filipe Manana (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (COUCHDB-1342) Asynchronous file writes
Date Sun, 22 Jan 2012 23:44:40 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-1342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13190821#comment-13190821
] 

Filipe Manana commented on COUCHDB-1342:
----------------------------------------

Made a few more tests comparing latest master against COUCHDB-1342 branch (after merging master
into it).

Database writes
=============

* 1Kb documents (https://github.com/fdmanana/basho_bench_couch/blob/master/couch_docs/doc_1kb.json)

http://graphs.mikeal.couchone.com/#/graph/7c13e2bdebfcd17aab424e68f225fe9a

* 2Kb documents (https://github.com/fdmanana/basho_bench_couch/blob/master/couch_docs/doc_2kb.json)

http://graphs.mikeal.couchone.com/#/graph/7c13e2bdebfcd17aab424e68f2261504

* 11Kb documents
(https://github.com/fdmanana/basho_bench_couch/blob/master/couch_docs/doc_11kb.json)

http://graphs.mikeal.couchone.com/#/graph/7c13e2bdebfcd17aab424e68f2262e98


View indexer
==========

Test database:  http://fdmanana.iriscouch.com/_utils/many_docs

* master

$ echo 3 > /proc/sys/vm/drop_caches
$ time curl http://localhost:5984/many_docs/_design/test/_view/test1
{"rows":[
{"key":null,"value":20000000}
]}

real	29m42.041s
user	0m0.016s
sys	0m0.036s


* master + patch (branch COUCHDB-1342)

$ echo 3 > /proc/sys/vm/drop_caches
$ time curl http://localhost:5984/many_docs/_design/test/_view/test1
{"rows":[
{"key":null,"value":20000000}
]}

real	26m13.112s
user	0m0.008s
sys	0m0.036s

Before COUCHDB-1334, and possibly the refactored indexer as well, the difference used to be
more significant (like in the results presented in the dev mail http://s.apache.org/TVu).
                
> Asynchronous file writes
> ------------------------
>
>                 Key: COUCHDB-1342
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-1342
>             Project: CouchDB
>          Issue Type: Improvement
>          Components: Database Core
>            Reporter: Jan Lehnardt
>             Fix For: 1.3
>
>         Attachments: COUCHDB-1342.patch
>
>
> This change updates the file module so that it can do
> asynchronous writes. Basically it replies immediately
> to process asking to write something to the file, with
> the position where the chunks will be written to the
> file, while a dedicated child process keeps collecting
> chunks and write them to the file (and batching them
> when possible). After issuing a series of write request
> to the file module, the caller can call its 'flush'
> function which will block the caller until all the
> chunks it requested to write are effectively written
> to the file.
> This maximizes the IO subsystem, as for example, while
> the updater is traversing and modifying the btrees and
> doing CPU bound tasks, the writes are happening in
> parallel.
> Originally described at http://s.apache.org/TVu
> Github Commit: https://github.com/fdmanana/couchdb/commit/e82a673f119b82dddf674ac2e6233cd78c123554

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message