couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jan Lehnardt (JIRA)" <j...@apache.org>
Subject [jira] Commented: (COUCHDB-532) Have requests for views that use 'stale=ok' return whats in the index immediately, but still kickoff background indexing process.
Date Sat, 17 Oct 2009 13:03:31 GMT

    [ https://issues.apache.org/jira/browse/COUCHDB-532?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12766904#action_12766904
] 

Jan Lehnardt commented on COUCHDB-532:
--------------------------------------

I think we have plans for a facility that fixes the underlying issue here without the proposed
way of solving it. Adam and I are dreaming up a "task manager" sorta like cron, that lets
you define arbitrary actions to be run on events, they could be time related, based on stats
counters or just continuous. for example continuous replication that is supposed to survive
a server restart is one of these things. or scheduling compaction to run each night at 2am.
another part is keeping views up to date asynchronously, so clients can use stale=ok all the
time. I'm pretty sure we a ticket for this, but I couldn't find it just now.

> Have requests for views that use 'stale=ok' return whats in the index immediately, but
still kickoff background indexing process.
> ---------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: COUCHDB-532
>                 URL: https://issues.apache.org/jira/browse/COUCHDB-532
>             Project: CouchDB
>          Issue Type: Improvement
>          Components: Database Core
>         Environment: All
>            Reporter: Glenn Rempe
>
> Currently a common use case for querying indexes from external applications is to pass
in the optional parameter 'stale=ok' which instructs the view to return what is has in its
index immediately without waiting for the index to first catch up.
> This is a *very* important feature as it allows applications to eliminate the poor user
experience of potential HTTP client timeouts or long'ish waits for a query to return while
the index is updated.
> However, applications that make extensive (or in my case exlusive) use of this feature
are now essentially *required* to build some kind of script which will do the indexing in
the background since no user requests will *ever* initiate indexing.  This would be for example
a cron script that makes an HTTP call to the view every N hours.  This becomes an operational
issue as you now have to write, maintain and schedule an external script which calls each
view in each of your DBs periodically.
> I propose to the team, after some discussion with Paul Joseph Davis, that view requests
with the 'stale=ok' param continue to return results in the as-is index to users immediately,
but *also* kick off the background indexing process.  This would have the effect of removing
the need for any external maintenance scripts (aside from DB/View compaction which is a related
but separate issue).
> In some use cases it might be true that indexing *must* occur outside of 'normal business
hours' due to processing load.  If this is a requirement the behavior of view indexing on
'stale' requests could be configurable (e.g. as a boolean to always/never do this additional
background indexing, or force BG indexing every N requests/minutes using 'stale').
> Cheers.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message