couchdb-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Kocoloski <kocol...@apache.org>
Subject Re: Indexes are inaccessible while indexers are running
Date Tue, 01 Nov 2016 16:08:13 GMT
Hi Graham, the indexes don’t update automatically, but it is possible to prime the indexers
by issuing a query to the view at any point during the import. One interesting option for
you is the "?stale=update_after" flag, which will respond with the current state of the view
index and trigger a background update of the indexes after the fact:

GET /<db>/_design/<ddoc>/_view/<view>?stale=update_after

You could also add a &limit=0 if you’re only interested in priming the indexers.

As far as indexing performance is concerned … ~4500 docs/second isn’t awesome, but the
devil is in the details: how many times is each document indexed? Does the server have adequate
CPU and IO? Are you running 2.0 or one of the 1.x versions?

I can dig up some benchmarks but I’m certain I’ve seen (Linux) systems index several times
faster than that. I haven’t seen a lot of extensive performance testing on Windows though.
Cheers,

Adam

> On Oct 31, 2016, at 7:41 AM, Graham Bull <calzakk+couchdb@gmail.com> wrote:
> 
> Hello,
> 
> I'm currently evaluating CouchDB (and other NoSQL databases).
> 
> I have a number of databases of various sizes. After restarting the CouchDB
> service (I'm on Windows) eight "indexer" tasks started running on the
> largest database (40 million documents), which was recently imported.
> 
> After 30 minutes the progress on all tasks is 20%. In the meantime I can't
> run any queries using the database's indexes. At this rate, it'll take
> around 2.5 hours to index the entire database.
> 
> Presumably, when indexes are created, they're initially empty? And the
> indexer tasks are required to do the actual indexing? If so, then the
> performance is pretty bad. It took nearly 2 hours to import the 40 million
> records. Add on index creation, and you're looking at 4.5 hours. Without
> mentioning other relational and NoSQL databases by name, or giving any
> stats, CouchDB's import and indexing performance is pretty bad in
> comparison.
> 
> Is there a way to force the indexers to run immediately after importing the
> data, and to query the indexing status so that my app can wait until it's
> completed?
> 
> Thanks in advance,
> 
> Graham


Mime
View raw message