lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From s1monw <...@git.apache.org>
Subject [GitHub] lucene-solr pull request #503: LUCENE-8571: Don't block on FrozenBufferedUpd...
Date Wed, 21 Nov 2018 09:31:11 GMT
GitHub user s1monw opened a pull request:

    https://github.com/apache/lucene-solr/pull/503

    LUCENE-8571: Don't block on FrozenBufferedUpdates#apply during IW#processEvents

    While indexing we try to apply frozen deletes packages concurrently
    on indexing threads if necessary. This is done in an opaque way via
    IndexWriter#processEvents. Yet, when we commit or refresh we have to
    ensure we apply all frozen update packages before we return.
    Today we execute the apply method in a blocking fashion which is unnecessary
    when we are in a IndexWriter#processEvents context, we block indexing
    threads while they could just continue since it's already being applied.
    We also might wait in BufferedUpdatesStream when we apply all necessary updates
    were we can continue with other work instead of waiting.
    This change also tries to apply the packages that are not currently applied
    first in order to not unnecessarily block.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/s1monw/lucene-solr optimize_apply_frozen_updates

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/lucene-solr/pull/503.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #503
    
----
commit caeb68509a53766963bbcb1a96ef204ba6cb1492
Author: Simon Willnauer <simonw@...>
Date:   2018-11-21T09:22:41Z

    LUCENE-8571: Don't block on FrozenBufferedUpdates#apply during IW#processEvents
    
    While indexing we try to apply frozen deletes packages concurrently
    on indexing threads if necessary. This is done in an opaque way via
    IndexWriter#processEvents. Yet, when we commit or refresh we have to
    ensure we apply all frozen update packages before we return.
    Today we execute the apply method in a blocking fashion which is unncessary
    when we are in a IndexWriter#processEvents context, we block indexing
    threads while they could just continue since it's already being applied.
    We also might wait in BufferedUpdatesStream when we apply all necessary updates
    were we can continue with other work instead of waiting.
    This change also tries to apply the packages that are not currently applied
    first in order to not unnecessarily block.

----


---

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message