couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Glenn Rempe (JIRA)" <j...@apache.org>
Subject [jira] Created: (COUCHDB-532) Have requests for views that use 'stale=ok' return whats in the index immediately, but still kickoff background indexing process.
Date Fri, 16 Oct 2009 22:49:31 GMT
Have requests for views that use 'stale=ok' return whats in the index immediately, but still
kickoff background indexing process.
---------------------------------------------------------------------------------------------------------------------------------

                 Key: COUCHDB-532
                 URL: https://issues.apache.org/jira/browse/COUCHDB-532
             Project: CouchDB
          Issue Type: Improvement
          Components: Database Core
         Environment: All
            Reporter: Glenn Rempe


Currently a common use case for querying indexes from external applications is to pass in
the optional parameter 'stale=ok' which instructs the view to return what is has in its index
immediately without waiting for the index to first catch up.

This is a *very* important feature as it allows applications to eliminate the poor user experience
of potential HTTP client timeouts or long'ish waits for a query to return while the index
is updated.

However, applications that make extensive (or in my case exlusive) use of this feature are
now essentially *required* to build some kind of script which will do the indexing in the
background since no user requests will *ever* initiate indexing.  This would be for example
a cron script that makes an HTTP call to the view every N hours.  This becomes an operational
issue as you now have to write, maintain and schedule an external script which calls each
view in each of your DBs periodically.

I propose to the team, after some discussion with Paul Joseph Davis, that view requests with
the 'stale=ok' param continue to return results in the as-is index to users immediately, but
*also* kick off the background indexing process.  This would have the effect of removing the
need for any external maintenance scripts (aside from DB/View compaction which is a related
but separate issue).

In some use cases it might be true that indexing *must* occur outside of 'normal business
hours' due to processing load.  If this is a requirement the behavior of view indexing on
'stale' requests could be configurable (e.g. as a boolean to always/never do this additional
background indexing, or force BG indexing every N requests/minutes using 'stale').

Cheers.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message