incubator-couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davis <paul.joseph.da...@gmail.com>
Subject Re: couchdb view performance
Date Fri, 27 Mar 2009 16:27:13 GMT
On Fri, Mar 27, 2009 at 2:50 AM, Adam Wolff <awolff@gmail.com> wrote:
> Hi paul,
> My system is being uncooporative - now I'm measuring 100-200ms response
> times, but I'm
> still pretty sure that couch is lagging.
> Anyway, I thought that maybe adding group=true when I queried the view with
> a key made a difference,
> but I wasn't sure. Should that make a difference? What does it mean to query
> with a key and without
> group=true?

You can only query a reduce by key if you have group=true, other wise
the default behavior is to produce a single row output. Using
group=true does make a difference as it needs to make a pass through
the JS VM per key (IIRC). This is what I mentioned about adding
optimizations later.

> I've attached my map/reduce. Thanks so much for all the help.

It looks like you're running up against the too much data problem in
that the value's you're returning from your reduce are growing too
fast, but its hard to tell without seeing the data.

> Adam
> On Thu, Mar 26, 2009 at 4:01 PM, Paul Davis <paul.joseph.davis@gmail.com>
> wrote:
>>
>> On Thu, Mar 26, 2009 at 6:36 PM, Adam Wolff <awolff@gmail.com> wrote:
>> > Hi everyone,
>> > We are running an alpha version of our software against a couchdb
>> > instance
>> > with a handful of documents, and we're seeing response times from our
>> > views
>> > of ~500ms. This is measured both within our application, and hitting the
>> > view directly using firebug+firefox.
>> > The view I'm talking about matches about 5 documents and returns about
>> > 9K of
>> > data. I'm running:
>> > Apache CouchDB 0.8.1-incubating (LogLevel=info)
>> > Erlang (BEAM) emulator version 5.6.5 [source] [async-threads:0] [hipe]
>> > [kernel-poll:false]
>> >
>> > This is all running on my MacBook Pro 2.33GHz Core 2 Duo with 3GB of
>> > RAM.
>> >
>>
>> You'll definitely want to upgrade to trunk, or 0.9 which is just now
>> out for testing pre-release at [1]. 500 ms is way way slow, trunk
>> should help, but there's probably something else going on as well.
>>
>> > By logging, I can see that my reduce function is running every time I
>> > access
>> > the view. The response time is about the same whether I've committed a
>> > new
>> > version of one of the documents in the view or not. This surprised me,
>> > since
>> > I thought that view results were cached. I've also tried logging the
>> > amount
>> > of time actually spent *in* my reduce function, but that appears to be
>> > negligible.
>> >
>>
>> The reduce function is generally run once per final reduce operation
>> currently. If I'm not mistaken, this means that you get it once per
>> key when group=true and just once when group=false
>>
>> > I am seeing some very fast responses from couchdb, for straight resource
>> > access -- on order 10ms. But all of my views are relatively slow -- even
>> > ones that don't have a reduce step.
>> >
>> > So, I'm wondering if I have a bad version, or bad config, or if this is
>> > expected performance. I'm sure things are running faster in trunk, but I
>> > want to get a feel for what kind of performance I can expect from a view
>> > with a fairly complicated reduce step.
>> >
>>
>> When you say fairly complicated, how do you mean? There is a size
>> output constraint for reductions. Ie, reduce functions should return
>> data that grows less than log(# keys reduced) because of data is
>> stored in the internal btree nodes.
>>
>> > Thanks in advance for any advice,
>> > A
>> >
>>
>> Also, the mechanics of reduce calculations have been on the back
>> burner for awhile in terms of keeping those partial reductions around.
>> I'm not 100% familiar with the entire code path, but I know that
>> there's definitely room for improvement but the speed optimizations
>> are being pushed back in favor of pulling in the big features.
>>
>> If nothing looks obvious, you can try pasting your M/R functions to
>> see if anyone spots something that looks slow.
>>
>> HTH,
>> Paul Davis
>
>

Mime
View raw message