couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Glenn Rempe (JIRA)" <>
Subject [jira] Commented: (COUCHDB-532) Have requests for views that use 'stale=ok' return whats in the index immediately, but still kickoff background indexing process.
Date Sat, 17 Oct 2009 16:48:31 GMT


Glenn Rempe commented on COUCHDB-532:

Jan,  This would, in my opinion, be an excellent new facility to have.  Based on your description
it would indeed solve the issue at hand nicely if it were time and event based.  Allowing
for cron-like control over all of the major aspects of CouchDB would be powerful.

Would you say this is a long term feature plan that is early in its design, or one that is
close to reaching fruition?  If its a longer term feature plan,  perhaps the enhancement to
do background indexing on stale=ok could be implemented as a stop-gap measure in the interim
if its easy to add (and just as easy to remove later)?  Speaking for myself I could use this
feature yesterday. :-)

> Have requests for views that use 'stale=ok' return whats in the index immediately, but
still kickoff background indexing process.
> ---------------------------------------------------------------------------------------------------------------------------------
>                 Key: COUCHDB-532
>                 URL:
>             Project: CouchDB
>          Issue Type: Improvement
>          Components: Database Core
>         Environment: All
>            Reporter: Glenn Rempe
> Currently a common use case for querying indexes from external applications is to pass
in the optional parameter 'stale=ok' which instructs the view to return what is has in its
index immediately without waiting for the index to first catch up.
> This is a *very* important feature as it allows applications to eliminate the poor user
experience of potential HTTP client timeouts or long'ish waits for a query to return while
the index is updated.
> However, applications that make extensive (or in my case exlusive) use of this feature
are now essentially *required* to build some kind of script which will do the indexing in
the background since no user requests will *ever* initiate indexing.  This would be for example
a cron script that makes an HTTP call to the view every N hours.  This becomes an operational
issue as you now have to write, maintain and schedule an external script which calls each
view in each of your DBs periodically.
> I propose to the team, after some discussion with Paul Joseph Davis, that view requests
with the 'stale=ok' param continue to return results in the as-is index to users immediately,
but *also* kick off the background indexing process.  This would have the effect of removing
the need for any external maintenance scripts (aside from DB/View compaction which is a related
but separate issue).
> In some use cases it might be true that indexing *must* occur outside of 'normal business
hours' due to processing load.  If this is a requirement the behavior of view indexing on
'stale' requests could be configurable (e.g. as a boolean to always/never do this additional
background indexing, or force BG indexing every N requests/minutes using 'stale').
> Cheers.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message