lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lance Norskog <goks...@gmail.com>
Subject Re: Multi-core memory problem
Date Thu, 08 Apr 2010 02:26:58 GMT
Sorting takes memory. What data types are the fields sorted on? If
they're strings, that could be a space-eater. If they are ints or
dates, not a problem.

Do the queries pull all of the documents found? Or do they just fetch
the, for example, first 10 documents?

What are the cache statistics like? Can they be shrunk? The stats are
shown the Statistics page off of the main solr/admin page.

Facets come from something called the Lucene Field Cache, which is not
controlled out of Solr. It has no eviction policy. When you do a facet
request, the memory used to load up the facets for a particular field
will not be evicted. So if you have lots and lots of facets, this
could be a problem.

On Wed, Apr 7, 2010 at 3:45 PM, Victoria Kagansky
<victoria.kagansky@gmail.com> wrote:
>  Hi,
> We are using Solr 1.4 running 2 cores each containing ~90M documents. Each
> core index size on the disk is ~ 120 G.
> The machine is a 64-bit quad-core 64G RAM running Windows Server 2008.
> Max heap size is set to 9G for the Tomcat process. Default caches are used.
>
> Our queries are complex and involve  8 facet fields (3 of them are boolean)
> and sorting on up 2 fields in addition to Solr score.
> I noticed a new behavior that didn't happened before: the first core being
> queried after startup answers all the queries, even the ones bringing ten
> millions of the documents, while the other core (that was queried the
> second) causes OutOfMemory exceptions for any query, even the "smallest"
> one. The heap is shown as not fully used 6-5 G out of 9. This is very
> strange because till recently both cores were working well, handling the
> heaviest queries, while the heap usage was on 8 G.
>
> Any ideas?
>
> Thanks
>



-- 
Lance Norskog
goksron@gmail.com

Mime
View raw message