lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rallavagu <>
Subject Re: Solr hard commit
Date Tue, 27 Oct 2015 15:56:12 GMT

On 10/27/15 8:43 AM, Erick Erickson wrote:
> bq: So, the updated file(s) on the disk automatically read into memory
> as they are Memory mapped?
> Not quite sure why you care, curiosity or is there something you're
> trying to accomplish?
This is out of curiosity. So, I can get better understanding of Solr's 
memory usage (heap & mmap).

> The contents of the index's segment files are read into virtual memory
> by MMapDirectory as needed to satisfy queries. Which is the point of
> autowarming BTW.

Ok. But, I have noticed that even "tlog" files are memory mapped (output 
from "lsof") in addition to all other files under "data" directory.

> commit in the following is either hard commit with openSearcher=true
> or soft commit.

Hard commit is setup with openSearcher=false and softCommit is setup for 
every 2 min.

> Segments that have been created (closed actually) after the last
> commit  are _not_ read at all until the next searcher is opened via
> another commit. Nothing is done with these new segments before the new
> searcher is opened which you control with your commit strategy.

I see. Thanks for the insight.

> Best,
> Erick
> On Mon, Oct 26, 2015 at 9:07 PM, Rallavagu <> wrote:
>> Erick, Thanks for clarification. I was under impression that MMapDirectory
>> is being used for both read/write operations. Now, I see how it is being
>> used. Essentially, it only reads from MMapDirectory and writes directly to
>> disk. So, the updated file(s) on the disk automatically read into memory as
>> they are Memory mapped?
>> On 10/26/15 8:43 PM, Erick Erickson wrote:
>>> You're really looking at this backwards. The MMapDirectory stuff is
>>> for Solr (Lucene, really) _reading_ data from closed segment files.
>>> When indexing, there are internal memory structures that are flushed
>>> to disk on commit, but these have nothing to do with MMapDirectory.
>>> So the question is really moot ;)
>>> Best,
>>> Erick
>>> On Mon, Oct 26, 2015 at 5:47 PM, Rallavagu <> wrote:
>>>> All,
>>>> Are memory mapped files (mmap) flushed to disk during "hard commit"? If
>>>> yes,
>>>> should we disable OS level (Linux for example) memory mapped flush?
>>>> I am referring to following for mmap files for Lucene/Solr
>>>> Linux level flush
>>>> Solr's hard and soft commit
>>>> Thanks in advance.

View raw message