incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randall Leeds <randall.le...@gmail.com>
Subject Re: Throughput when include_docs is true
Date Tue, 30 Mar 2010 17:44:57 GMT
When you say 3% io usage, what are you measuring?
3% of theoretical throughput on drive read?

What I'd be most interested in is something like the %wait metric from
the iostat command to know how much couchdb is waiting for your disk.
Without knowing what metric the 3% comes from it's hard to know if
there is a lot of seek wait time happening or not.

On Tue, Mar 30, 2010 at 18:29, Chris Stockton <chrisstocktonaz@gmail.com> wrote:
> Hello,
>
> Recently we were having a issue with our index being many gigabytes
> larger then our database. We were indexing the entire document, this
> has a great performance increase, on a 8 core xeon machine with 50gigz
> of ram calling a 450K doc view that is 80MB worth of raw data over the
> wire we get roughly 5MB worth of throughput when calling our view.
>
> However because of the unrealistic index sizes (after compaction) we
> were unable to continue storing the document within the index. Now we
> use the include_docs flag, but it seems this comes with a great
> performance penalty. On the same data & view (demote of storing the
> doc within the view) we are capped at about 400KB per second. For each
> additional call on the same view we end up sharing that 400KB of
> throughput, I.E. 4 connections get 100KB each, 8, 50KB each. Something
> i found interesting was we are NOT iobound at all, with 4 connections
> running we run at about 290% cpu usage, but at only 3% io. That is
> some pretty heavy cpu usage.
>
> Has anyone came across any tricks or optimizations for this? I would
> greatly appreciate any feedback.
>
> -Chris
>

Mime
View raw message