jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thomas Mueller (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OAK-8343) Allow queries to be delayed until an index is available
Date Wed, 29 May 2019 07:08:00 GMT

    [ https://issues.apache.org/jira/browse/OAK-8343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16850564#comment-16850564

Thomas Mueller commented on OAK-8343:

The AsyncIndexerService can be used to register async lanes (thanks [~catholicon]).

There is a problem if the sequence is as follows:
* 1: async lanes are registered early, e.g. using AsyncIndexerService
* 2: a query is run that might the async index "x"
* 3: the thread pool started, so async indexing can occur
* 4: a query is run that actually needs the async index "x"

 Now if we set waitForIndex for index "x", then Oak would wait at 2, which is wrong.

> Allow queries to be delayed until an index is available
> -------------------------------------------------------
>                 Key: OAK-8343
>                 URL: https://issues.apache.org/jira/browse/OAK-8343
>             Project: Jackrabbit Oak
>          Issue Type: Improvement
>          Components: lucene, query
>            Reporter: Thomas Mueller
>            Assignee: Thomas Mueller
>            Priority: Major
>             Fix For: 1.14.0
>         Attachments: OAK-8343-b.patch, OAK-8343.patch
> Currently, indexes are built asynchronously. That is, if an index definition is added,
the index is eventually built, but it's quite hard to say when it is ready for queries. This
can be specially a problem right after the initial repository initialization, or after an
> In theory, system startup could be delayed until all indexes are ready (e.g. set the
"reindex" flag for important indexes, and at startup, wait until the "reindex" flag is set
to "false"). However, doing that would block threads that _don't_ need an index. It would
be better to only block threads that actually do run queries. That would make startup deterministic,
without delaying other threads unnecessarily.
> To solve the problem, we can add a property "waitForIndex" in the index definition (just
Lucene indexes is fine for now, as those are the important asynchronous ones). If set, then
queries that potentially use those indexes are delayed, until the indexes are ready for sure.
Reindex would need to remove that property (the same as it removes e.g. refresh or sets reindex
to false). For added security, queries are only blocked as long as "reindex" is also set to
true (this ensures that waitForIndex is removed eventually), and waiting should time out after
2 minutes, to ensure the feature doesn't block startup forever if indexing fails for some

This message was sent by Atlassian JIRA

View raw message