couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Stockton <>
Subject Throughput when include_docs is true
Date Tue, 30 Mar 2010 17:29:39 GMT

Recently we were having a issue with our index being many gigabytes
larger then our database. We were indexing the entire document, this
has a great performance increase, on a 8 core xeon machine with 50gigz
of ram calling a 450K doc view that is 80MB worth of raw data over the
wire we get roughly 5MB worth of throughput when calling our view.

However because of the unrealistic index sizes (after compaction) we
were unable to continue storing the document within the index. Now we
use the include_docs flag, but it seems this comes with a great
performance penalty. On the same data & view (demote of storing the
doc within the view) we are capped at about 400KB per second. For each
additional call on the same view we end up sharing that 400KB of
throughput, I.E. 4 connections get 100KB each, 8, 50KB each. Something
i found interesting was we are NOT iobound at all, with 4 connections
running we run at about 290% cpu usage, but at only 3% io. That is
some pretty heavy cpu usage.

Has anyone came across any tricks or optimizations for this? I would
greatly appreciate any feedback.


View raw message