lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: Solr using a ridiculous amount of memory
Date Fri, 19 Apr 2013 11:30:44 GMT
Hmmm. There has been quite a bit of work lately to support a couple of
things that might be of interest (4.3, which Simon cut today, probably
available to all mid next week at the latest). Basically, you can
choose to pre-define all the cores in solr.xml (so-called "old style")
_or_ use the new-style solr.xml which uses "auto-discover" mode to
walk the indicated directory and find all the cores (indicated by the
presence of a 'core.properties' file). Don't know if this would make
your particular case easier, and I should warn you that this is
relatively new code (although there are some reasonable unit tests).

You also have the option to only load the cores when they are
referenced, and only keep N cores open at a time (loadOnStartup and
transient properties).

See: http://wiki.apache.org/solr/CoreAdmin#Configuration and
http://wiki.apache.org/solr/Solr.xml%204.3%20and%20beyond

Note, the docs are somewhat sketchy, so if you try to go down this
route let us know anything that should be improved (or you can be
added to the list of wiki page contributors and help out!)

Best
Erick

On Thu, Apr 18, 2013 at 8:31 AM, John Nielsen <jn@mcb.dk> wrote:
>> You are missing an essential part: Both the facet and the sort
>> structures needs to hold one reference for each document
>> _in_the_full_index_, even when the document does not have any values in
>> the fields.
>>
>
> Wow, thank you for this awesome explanation! This is where the penny
> dropped for me.
>
> I will definetely move to a multi-core setup. It will take some time and a
> lot of re-coding. As soon as I know the result, I will let you know!
>
>
>
>
>
>
> --
> Med venlig hilsen / Best regards
>
> *John Nielsen*
> Programmer
>
>
>
> *MCB A/S*
> Enghaven 15
> DK-7500 Holstebro
>
> Kundeservice: +45 9610 2824
> post@mcb.dk
> www.mcb.dk

Mime
View raw message