Mailing-List: contact user-help@couchdb.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@couchdb.apache.org
Received-SPF: pass (nike.apache.org: domain of rgabo@rgabostyle.com designates
 209.85.161.52 as permitted sender)
Date: Wed, 8 Jun 2011 10:56:39 +0200
From: Gabor Ratky <rgabo@rgabostyle.com>
To: user@couchdb.apache.org
Message-ID: <ECE405B7561E4AC196596899BBB459A5@rgabostyle.com>
In-Reply-To: <BANLkTi=eTFX67LgFT3g3W9iihG0N5koC7Q@mail.gmail.com>
References: <4DDDFD4E.9060701@gmail.com>
 <BANLkTimxsnRiAJh=+c=YWSbYxv1ba51e_w@mail.gmail.com>
 <4DDF164A.7030009@gmail.com>
 <FF823898F63A415F8CCB2EB8D4D2B48D@rgabostyle.com>
 <4DDF6BEE.8070907@gmail.com> <4DE4F6F4.9000406@couchone.com>
 <4DECC426.9070201@gmail.com>
 <BANLkTimeE_jR0cKaSubn96b64tisq-FB7g@mail.gmail.com>
 <4DEDC36C.5090207@gmail.com> <4DEEFF3E.1070807@gmail.com>
 <BANLkTik8882gSVaMYhTq4JmNSuypb9aL3A@mail.gmail.com>
 <BANLkTi=Ox94g=ZjjhrZY8i=uovFuvWLefA@mail.gmail.com>
 <BANLkTi=eTFX67LgFT3g3W9iihG0N5koC7Q@mail.gmail.com>
Subject: Re: Complex queries & results
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="4def3947_52d7b105_108"

--4def3947_52d7b105_108
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

The default blocking also only applies to views in a single design document, any other view in other design documents can be queried and will respond immediately if they are up to date (or with stale=ok, stale=update_after as mentioned below). This also means that the view update is triggered only for views in the design document in question and not others.

Gabor

On Wednesday, June 8, 2011 at 10:53 AM, Robert Newson wrote:

> "couch doesn't answer any query while it's doing its view updates."
> 
> This is not true, though it is the default behavior to block queries
> until the view is up to date. Pass ?stale=ok to get results from the
> view immediately (but not that the view won't be updated). See the new
> ?stale=update_after for a non-blocking view query that triggers a view
> update asynchronously.
> 
> B.
> 
> On 8 June 2011 06:56, Marcello Nuccio <marcello.nuccio@gmail.com (mailto:marcello.nuccio@gmail.com)> wrote:
> > I have not tested it, but CouchDB 1.1 does have some improvements:
> > 
> > http://docs.couchbase.org/couchdb-release-1.1/index.html#couchdb-release-1.1-etag
> > http://docs.couchbase.org/couchdb-release-1.1/index.html#coudhdb-release-1.1-updateafter
> > 
> > - Marcello
> > 
> > 2011/6/8 Mark Hahn <mark@boutiquing.com (mailto:mark@boutiquing.com)>:
> > > > couch doesn't answer any query while it's doing its view updates.
> > > 
> > > I was shocked when I first experienced this myself. I posted here and found
> > > the bad news. This was the first and so far only serious wart I've found on
> > > couch.
> > > 
> > > I received replies suggesting that I put different views in separate design
> > > docs. Apparently the blocking only happens on the same design doc, at least
> > > when updating the view code. When one view in a design doc needs updating
> > > it updates them all. I haven't tried this yet.
> > > 
> > > I also haven't yet played with the "stale" option, which allows reading docs
> > > that are out-of-date. Between the two options I've got my fingers crossed
> > > that I can avoiding any real blocking. Blocking a server is the worst sin
> > > of all.
> > > 
> > > 
> > > On Tue, Jun 7, 2011 at 9:49 PM, Torstein Krause Johansen <
> > > torsteinkrausework@gmail.com (mailto:torsteinkrausework@gmail.com)> wrote:
> > > 
> > > > On 07/06/11 14:21, Torstein Krause Johansen wrote:
> > > > 
> > > > On 06/06/11 22:09, Sean Copenhaver wrote:
> > > > 
> > > > https://gist.github.com/1010318
> > > > > > 
> > > > > > I tried this out with 10 docs fitting your example structure and with a
> > > > > > plain query (no grouping, no filtering, reduce on) I get back:
> > > > > > 
> > > > > > { John: 4, Jane: 6 }
> > > > > 
> > > > > Looks spot on! Thank you _so_ much for doing this.
> > > > > 
> > > > > I'm really curious how this performs, I will be-siege my couch with bulk
> > > > > updates, giving it a big-ish data set while simultaneously be-siege it
> > > > > with reads GETs querying this map/reduce you've created. Will be very
> > > > > interesting.
> > > > 
> > > > I started by using siege to post 1000s of documents with 14 fields &
> > > > values (the actual data my application will be using) and let it run
> > > > till I got a fair data set. After reducing the now ~710,000 document big DB
> > > > from 4.2GB to ~360MB, the queries went from ~8s to ~0.05s. Fantastic.
> > > > 
> > > > I then unleashed siege again (100 parallel threads this time, creating
> > > > 200 new documents each using the bulk endpoint (siege somehow didn't
> > > > want to work with my initial 1000 document big .json file, so I had to
> > > > reduce it to 200 to make siege not choke on it)) and wget (creating random
> > > > data, using the normal document endpoint), the queries immediately started
> > > > to climb upwards, 1s, 2s, 3s ... 80s and with no sign of stopping.
> > > > 
> > > > To see if it was the simultaneous write and read that were causing the
> > > > longer query times, I stopped siege and wget on my test machine
> > > > (different host, going through the same network switch).
> > > > 
> > > > Since there had been quite a number of new documents, couch started
> > > > its checkpoint view updating leaving my couch unable to respond to any
> > > > queries for around 90s.
> > > > 
> > > > The query times then dropped down, stabilising on 0.06 to 0.08s when
> > > > querying the DB with now ~800,000 documents and result sets containing ~50
> > > > keys with ~2000 counts each. Great!
> > > > 
> > > > The climbing query times when doing so many updates is not a real
> > > > concern for me as I'll put a queue in front of couch which buffers up
> > > > the incoming write requests and fires up a bulk update every 30
> > > > seconds or so. Couch seems more than fast enough write-wise as long as
> > > > the documents are provided in bulks.
> > > > 
> > > > What does worry me, though, is that couch doesn't answer any query
> > > > while it's doing its view updates. Even with a nice cache server in
> > > > front which can serve old content till couch is finished updating its
> > > > views, I still find it a bit unsettling. Do you have any tips for me here?
> > > > 
> > > > Cheers,
> > > > 
> > > > -Torstein
> > > 
> > > 
> > > 
> > > --
> > > Mark Hahn
> > > Website Manager
> > > mark@boutiquing.com (mailto:mark@boutiquing.com)
> > > 949-229-1012


--4def3947_52d7b105_108--