lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James liu" <liuping.ja...@gmail.com>
Subject Re: solr index problem
Date Thu, 19 Jul 2007 00:48:09 GMT
I correct it,,,i index 17M docs. not 1.7M,,,so OutOfMemory happen when it
finish index ~11.3m docs

It is new index.

i think it maybe the reason:

On 7/18/07, Otis Gospodnetic <otis_gospodnetic@yahoo.com> wrote:
> > Why?  Too small of a Java heap. :)
> > Increase the size of the Java heap and lower the maxBufferedDocs number
> in solrconfig.xml and then try again.
>
> If it only happens after a lot of docs, it's probably not
> maxBufferedDocs, but when a big luicene merge is triggered.
>


2007/7/19, Mike Klaas <mike.klaas@gmail.com>:
>
>
> On 18-Jul-07, at 2:58 PM, Yonik Seeley wrote:
>
> > On 7/18/07, Mike Klaas <mike.klaas@gmail.com> wrote:
> >>
> >> Could happen when doDeleting the pending docs too.  James: try
> >> sending commit every 500k docs or so.
> >
> > Hmmm, right... some of the memory usage will be related to the treemap
> > keeping track of deleted items, and some of it will be related to the
> > reader itself (the term index, and the norms).
> >
> > Perhaps we should have some sort of limit on the number of deletions
> > we cache so people don't have to worry about that part.
>
> I logged this in http://issues.apache.org/jira/browse/SOLR-310
>
> Perhaps a solrconfig.xml setting, <maxPendingDocs>? Default to 100k?
>
> I have experienced this from time to time... it is one of the reasons
> I fastidiously wipe the index and restart solr before reindexing.


aha,,me too。

-Mike
>



-- 
regards
jl
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message