Mailing-List: contact solr-user-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: solr-user@lucene.apache.org
Subject: Re: Suggester uses lots of 'Page cache' memory
To: solr-user@lucene.apache.org
References: <CAC9-YrUN9=TqkPQYwP-NkBkLQZmDryMCyQJiu_9JJbhHxcD_hg@mail.gmail.com>
From: Shawn Heisey <apache@elyograg.org>
Message-ID: <c107e7cb-96f4-ac24-7f1d-8b8b470ef855@elyograg.org>
Date: Tue, 2 May 2017 07:07:21 -0600
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101
 Thunderbird/45.8.0
MIME-Version: 1.0
In-Reply-To: <CAC9-YrUN9=TqkPQYwP-NkBkLQZmDryMCyQJiu_9JJbhHxcD_hg@mail.gmail.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 02 May 2017 13:07:40 -0000

On 5/1/2017 10:52 PM, Damien Kamerman wrote:
> I have a Solr v6.4.2 collection with 12 shards and 2 replicas. Each
> replica uses about 14GB disk usage. I'm using Solaris 11 and I see the
> 'Page cache' grow by about 7GB for each suggester replica I build. The
> suggester index itself is very small. The 'Page cache' memory is freed
> when the node is stopped. I guess the Suggester component is mmap'ing
> the entire Lucene index into memory and holding it? Is this expected
> behavior? Is there a workaround? 

I found the following.  The last comment on the answer, the one about
mmap causing double-buffering with ZFS, is possibly relevant:

https://serverfault.com/a/270604

What filesystem are your indexes on?  If it's ZFS, it could completely
explain the behavior.  If it's not ZFS, then the only part of it that I
cannot explain is the fact that the page cache is freed when Solr stops.

If this double-buffering actually means that the memory is allocated
twice, then I think that ZFS is probably the wrong filesystem to run
Solr on, unless you have a LOT of spare memory.  You could try changing
the directory factory to one that doesn't use MMAP, but the suggester
index factory probably cannot be easily changed.  This is too bad --
normally MMAP is far more efficient than "standard" filesystem access.

I could be reaching completely wrong conclusions based on the limited
research I did.

Thanks,
Shawn