lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Torsten Bøgh Köster (JIRA) <>
Subject [jira] [Updated] (SOLR-10506) Possible memory leak upon collection reload
Date Tue, 18 Apr 2017 10:17:41 GMT


Torsten Bøgh Köster updated SOLR-10506:
    Attachment: solr_gc_path_via_zk_WatchManager.png

The strong references via schema index reader

> Possible memory leak upon collection reload
> -------------------------------------------
>                 Key: SOLR-10506
>                 URL:
>             Project: Solr
>          Issue Type: Bug
>      Security Level: Public(Default Security Level. Issues are Public) 
>          Components: Server
>    Affects Versions: 6.5
>            Reporter: Torsten Bøgh Köster
>         Attachments: solr_collection_reload_13_cores.png, solr_gc_path_via_zk_WatchManager.png
> Upon manual Solr Collection reloading, references to the closed {{SolrCore}} are not
fully removed by the garbage collector as a strong reference to the {{ZkIndexSchemaReader}}
is held in a ZooKeeper {{Watcher}} that watches for schema changes.
> In our case, this leads to a massive memory leak as managed resources are still referenced
by the closed {{SolrCore}}. Our Solr cloud environment utilizes rather large managed resources
(synonyms, stopwords). To reproduce, we fired out environment up and reloaded the collection
13 times. As a result we fully exhausted our heap. A closer look with the Yourkit profiler
revealed 13 {{SolrCore}} instances, still holding strong references to the garbage collection
root (see screenshot 1).
> Each {{SolrCore}} instance holds a single path with strong references to the gc root
via a `Watcher` in `ZkIndexSchemaReader` (see screenshot 2). The {{ZkIndexSchemaReader}} registers
a close hook in the {{SolrCore}} but the Zookeeper is not removed upon core close.
> We supplied a Github Pull Request that extracts the zookeeper `Watcher` as a static inner
class. To eliminate the memory leak, the schema reader is held inside a `WeakReference` and
the reference is explicitly removed on core close.
> Initially I wanted to supply a test case but unfortunately did not find a good starting
point ...

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message