lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tboeghk <...@git.apache.org>
Subject [GitHub] lucene-solr pull request #197: [SOLR-10506] Fixes a memory leak in zk schema...
Date Thu, 04 May 2017 11:26:36 GMT
GitHub user tboeghk opened a pull request:

    https://github.com/apache/lucene-solr/pull/197

    [SOLR-10506] Fixes a memory leak in zk schema watching

    Upon manual Solr Collection reloading, references to the closed SolrCore are not fully
removed by the garbage collector as a strong reference to the ZkIndexSchemaReader is held
in a ZooKeeper Watcher that watches for schema changes.
    
    In our case, this leads to a massive memory leak as managed resources are still referenced
by the closed SolrCore. Our Solr cloud environment utilizes rather large managed resources
(synonyms, stopwords). To reproduce, we fired out environment up and reloaded the collection
13 times. As a result we fully exhausted our heap. A closer look with the Yourkit profiler
revealed 13 SolrCore instances, still holding strong references to the garbage collection
root.
    
    Each SolrCore instance holds a single path with strong references to the gc root via a
Watcher in ZkIndexSchemaReader. The ZkIndexSchemaReader registers a close hook in the SolrCore
but the Zookeeper is not removed upon core close.
    
    We supplied this Github Pull Request that extracts the zookeeper Watcher as a static inner
class. To eliminate the memory leak, the schema reader is held inside a WeakReference and
the reference is explicitly removed on core close.
    
    Initially I wanted to supply a test case but unfortunately did not find a good starting
point ...
    
    N.B.: I did this second PR for the same issue to separate code changes for both SOLR-10506
and SOLR-10550 which I maintained on the same fork branch :-/

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/shopping24/lucene-solr branch_6_5__SOLR-10506

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/lucene-solr/pull/197.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #197
    
----
commit aa3a96cd88d1724f3ab641f25c5373ff58f4616d
Author: Torsten Bøgh Köster <torsten.koester@s24.com>
Date:   2017-04-18T09:43:52Z

    Fixes a memory leak in zk schema watching

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message