Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (athena.apache.org: domain of robert.newson@gmail.com
 designates 209.85.214.52 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :content-type:content-transfer-encoding;
        b=PDzOw5pHKsAMty0Ca3WrunRmlu4qpF6S8foXqAIbduTrOMYDqx+SodrfyEWmtLp3vT
         mI5RQUoxW2LEcyhMQF4AVTkeWYR8Xm0SWDZ3I89bhc/1zcg/g4APrPnQZIWvM47rSX/s
         +hrekricrvW70ndvdR+T+oVdka8yVjggVbch4=
MIME-Version: 1.0
In-Reply-To: <BANLkTi=Ox94g=ZjjhrZY8i=uovFuvWLefA@mail.gmail.com>
References: <4DDDFD4E.9060701@gmail.com>
	<BANLkTimxsnRiAJh=+c=YWSbYxv1ba51e_w@mail.gmail.com>
	<4DDF164A.7030009@gmail.com>
	<FF823898F63A415F8CCB2EB8D4D2B48D@rgabostyle.com>
	<4DDF6BEE.8070907@gmail.com>
	<4DE4F6F4.9000406@couchone.com>
	<4DECC426.9070201@gmail.com>
	<BANLkTimeE_jR0cKaSubn96b64tisq-FB7g@mail.gmail.com>
	<4DEDC36C.5090207@gmail.com>
	<4DEEFF3E.1070807@gmail.com>
	<BANLkTik8882gSVaMYhTq4JmNSuypb9aL3A@mail.gmail.com>
	<BANLkTi=Ox94g=ZjjhrZY8i=uovFuvWLefA@mail.gmail.com>
Date: Wed, 8 Jun 2011 09:53:45 +0100
Message-ID: <BANLkTi=eTFX67LgFT3g3W9iihG0N5koC7Q@mail.gmail.com>
Subject: Re: Complex queries & results
From: Robert Newson <robert.newson@gmail.com>
To: user@couchdb.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

"couch doesn't answer any query while it's doing its view updates."

This is not true, though it is the default behavior to block queries
until the view is up to date. Pass ?stale=3Dok to get results from the
view immediately (but not that the view won't be updated). See the new
?stale=3Dupdate_after for a non-blocking view query that triggers a view
update asynchronously.

B.

On 8 June 2011 06:56, Marcello Nuccio <marcello.nuccio@gmail.com> wrote:
> I have not tested it, but CouchDB 1.1 does have some improvements:
>
> http://docs.couchbase.org/couchdb-release-1.1/index.html#couchdb-release-=
1.1-etag
> http://docs.couchbase.org/couchdb-release-1.1/index.html#coudhdb-release-=
1.1-updateafter
>
> - Marcello
>
> 2011/6/8 Mark Hahn <mark@boutiquing.com>:
>>> couch doesn't answer any query while it's doing its view updates.
>>
>> I was shocked when I first experienced this myself. =A0I posted here and=
 found
>> the bad news. =A0This was the first and so far only serious wart I've fo=
und on
>> couch.
>>
>> I received replies suggesting that I put different views in separate des=
ign
>> docs. =A0Apparently the blocking only happens on the same design doc, at=
 least
>> when updating the view code. =A0When one view in a design doc needs upda=
ting
>> it updates them all. =A0I haven't tried this yet.
>>
>> I also haven't yet played with the "stale" option, which allows reading =
docs
>> that are out-of-date. =A0Between the two options I've got my fingers cro=
ssed
>> that I can avoiding any real blocking. =A0Blocking a server is the worst=
 sin
>> of all.
>>
>>
>> On Tue, Jun 7, 2011 at 9:49 PM, Torstein Krause Johansen <
>> torsteinkrausework@gmail.com> wrote:
>>
>>> On 07/06/11 14:21, Torstein Krause Johansen wrote:
>>>
>>> =A0On 06/06/11 22:09, Sean Copenhaver wrote:
>>>>
>>>
>>> =A0https://gist.github.com/1010318
>>>>>
>>>>> I tried this out with 10 docs fitting your example structure and with=
 a
>>>>> plain query (no grouping, no filtering, reduce on) I get back:
>>>>>
>>>>> { John: 4, Jane: 6 }
>>>>>
>>>>
>>>> Looks spot on! Thank you _so_ much for doing this.
>>>>
>>>> I'm really curious how this performs, I will be-siege my couch with bu=
lk
>>>> updates, giving it a big-ish data set while simultaneously be-siege it
>>>> with reads GETs querying this map/reduce you've created. Will be very
>>>> interesting.
>>>>
>>>
>>> I started by using siege to post 1000s of documents with 14 fields &
>>> values (the actual data my application will be using) and let it run
>>> till I got a fair data set. After reducing the now ~710,000 document bi=
g DB
>>> from 4.2GB to ~360MB, the queries went from ~8s to ~0.05s. Fantastic.
>>>
>>> I then unleashed siege again (100 parallel threads this time, creating
>>> 200 new documents each using the bulk endpoint (siege somehow didn't
>>> want to work with my initial 1000 document big .json file, so I had to
>>> reduce it to 200 to make siege not choke on it)) and wget (creating ran=
dom
>>> data, using the normal document endpoint), the queries immediately star=
ted
>>> to climb upwards, 1s, 2s, 3s ... 80s and with no sign of stopping.
>>>
>>> To see if it was the simultaneous write and read that were causing the
>>> longer query times, I stopped siege and wget on my test machine
>>> (different host, going through the same network switch).
>>>
>>> Since there had been quite a number of new documents, couch started
>>> its checkpoint view updating leaving my couch unable to respond to any
>>> queries for around 90s.
>>>
>>> The query times then dropped down, stabilising on 0.06 to 0.08s when
>>> querying the DB with now ~800,000 documents and result sets containing =
~50
>>> keys with ~2000 counts each. Great!
>>>
>>> The climbing query times when doing so many updates is not a real
>>> concern for me as I'll put a queue in front of couch which buffers up
>>> the incoming write requests and fires up a bulk update every 30
>>> seconds or so. Couch seems more than fast enough write-wise as long as
>>> the documents are provided in bulks.
>>>
>>> What does worry me, though, is that couch doesn't answer any query
>>> while it's doing its view updates. Even with a nice cache server in
>>> front which can serve old content till couch is finished updating its
>>> views, I still find it a bit unsettling. Do you have any tips for me he=
re?
>>>
>>> Cheers,
>>>
>>> -Torstein
>>>
>>
>>
>>
>> --
>> Mark Hahn
>> Website Manager
>> mark@boutiquing.com
>> 949-229-1012
>>
>