couchdb-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Apache Wiki <wikidi...@apache.org>
Subject [Couchdb Wiki] Update of "Performance" by RandallLeeds
Date Thu, 17 Jun 2010 20:34:01 GMT
Dear Wiki user,

You have subscribed to a wiki page or wiki category on "Couchdb Wiki" for change notification.

The "Performance" page has been changed by RandallLeeds.
http://wiki.apache.org/couchdb/Performance?action=diff&rev1=3&rev2=4

--------------------------------------------------

  
  As an example one of my benchmarks turned out to be mostly limited by the json module's
encoding and decoding speed.  The process was using 40% of a CPU.  Switching to simplejson
with no other changes resulted in 5% of a CPU.  Switching from threads to processes (using
multiprocessing module) gave yet another performance improvement finally pushing CouchDB to
consume more than 100% of a CPU (this is on a multi-processor machine).
  
+ = Resource Limits =
+ One of the problems that administrators run into as their deployments become large are resource
limits imposed by the system and by the application configuration. Raising these limits can
allow your deployment to grow beyond what the default configuration will support.
+ == CouchDB Configuration Options ==
+ In your configuration (local.ini or similar) familiarize yourself with the following options:{{{
+ [couchdb]
+ max_dbs_open = 100
+ 
+ [httpd]
+ max_connections = 2048}}}
+ The first option places an upper bound on the number of databases that can be open at one
time. CouchDB reference counts database accesses internally and will close idle databases
when it must. Sometimes it is necessary to keep more than the default open at once, such as
in deployments where many databases will be continuously replicating.
+ The second option limits how many client connections the HTTP server will service at a time.
Again, heavy replication scenarios are good candidates for increased {{{max_connections}}}
since the replicator opens several connections to the source database.
+ == System Resource Limits ==
+ === Erlang ===
+ Even if you've increased the maximum connections CouchDB will allow, the Erlang runtime
system will not allow more than 1024 connections by default. Setting the following option
in {{{(prefix)/etc/default/couchdb}}} (or equivalent) will increase this limit (in this case
to 4096): {{{export ERL_MAX_PORTS=4096}}}
+ === PAM and ulimit ===
+ Finally, most *nix operating systems impose various resource limits on every process. If
your system is set up to use the Pluggable Authentication Modules (PAM) system, increasing
this limit is straightforward. For example, creating a file named {{{/etc/security/limits.d/100-couchdb.conf}}}
with the following contents will ensure that CouchDB can open enough file descriptors to service
your increased maximum open databases and Erlang ports:{{{
+ #<domain>    <type>    <item>    <value>
+ couchdb      hard      nofile    4096
+ couchdb      soft      nofile    4096}}}
+ If your system does not use PAM, a {{{ulimit}}} command is usually available for use in
a custom script to launch CouchDB with increased resource limits.
+ If necessary, feel free to increase this limits as long as your hardware can handle the
load.
+ 

Mime
View raw message