lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Crump <jeffrey.cr...@gmail.com>
Subject Re: Out of memory during the indexing
Date Fri, 02 Dec 2011 19:26:53 GMT
Can anyone advise techniques for limiting the size of the RAM buffers to
begin with?  As the index grows, I shouldn't have to keep increasing the
heap.  We have a high-ingest, low-query-rate environment and I'm not as
much concerned with the query-time caches as I am with the segment core
readers/SolrIndexSearchers themselves.

On 9 November 2011 06:10, Andre Bois-Crettez <andre.bois@kelkoo.com> wrote:

> How much memory you actually allocate to the JVM ?
> http://wiki.apache.org/solr/**SolrPerformanceFactors#Memory_**
> allocated_to_the_Java_VM<http://wiki.apache.org/solr/SolrPerformanceFactors#Memory_allocated_to_the_Java_VM>
> You need to increase the -Xmx value, otherwise your large ram buffers
> won't fit in the java heap.
>
>
>
> sivaprasad wrote:
>
>> Hi,
>>
>> I am getting the following error during the indexing.I am trying to index
>> 14
>> million records but the document size is very minimal.
>>
>> *Error:*
>> 2011-11-08 14:53:24,634 ERROR [STDERR] (Thread-12)
>> java.lang.OutOfMemoryError: GC overhead limit exceeded
>>
>>
>>
> [...]
>
>  Do i need to increase the heap size for JVM?
>>
>> My solrconfig settings are given below.
>>
>> <indexDefaults>
>>      <useCompoundFile>false</**useCompoundFile>
>>
>>    <mergeFactor>25</mergeFactor>
>>        <maxBufferedDocs>2</**maxBufferedDocs>
>>       <ramBufferSizeMB>1024</**ramBufferSizeMB>
>>    <maxMergeDocs>2147483647</maxMergeDocs>
>>    <maxFieldLength>10000</**maxFieldLength>
>>    <writeLockTimeout>1000</**writeLockTimeout>
>>    <commitLockTimeout>10000</**commitLockTimeout>
>>
>> and the main index values are
>> <useCompoundFile>false</**useCompoundFile>
>>    <ramBufferSizeMB>512</**ramBufferSizeMB>
>>    <mergeFactor>10</mergeFactor>
>>    <maxMergeDocs>2147483647</maxMergeDocs>
>>    <maxFieldLength>10000</**maxFieldLength>
>>
>> Do i need to increase the ramBufferSizeMB to a little higher?
>>
>> Please provide your inputs.
>>
>> Regards,
>> Siva
>>
>> --
>> View this message in context: http://lucene.472066.n3.**
>> nabble.com/Out-of-memory-**during-the-indexing-**tp3492701p3492701.html<http://lucene.472066.n3.nabble.com/Out-of-memory-during-the-indexing-tp3492701p3492701.html>
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>>
>>
>
> --
> André Bois-Crettez
>
> Search technology, Kelkoo
> http://www.kelkoo.com/
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message