lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Li Li <fancye...@gmail.com>
Subject Re: New segment file created too often
Date Mon, 13 Feb 2012 17:30:09 GMT
as far as I know, there are three situation it will be flushed to a new
segment: RAM buffer for posting data structure is used up; added doc
numbers are exceeding threshold and there are many deletions in a segment
but your configuration seems it is not likely to flush many small segments.

<ramBufferSizeMB>1024</ramBufferSizeMB>
<maxMergeDocs>2147483647</maxMergeDocs>
On Tue, Feb 14, 2012 at 1:10 AM, Huy Le <huyle@springpartners.com> wrote:

> Hi,
>
> I am using solr 3.5.  As I understood it, NRT is a solr 4 feature, but solr
> 4 is not released yet.
>
> I understand commit after adding each document is expensive, but the
> application requires that documents be available after adding to the index.
>
> What I don't understand is why new segment files are created so often.
> Are the commit calls triggering new segment files being created?  I don't
> see this behavior in another environment of the same version of solr.
>
> Huy
>
> On Mon, Feb 13, 2012 at 11:55 AM, Li Li <fancyerii@gmail.com> wrote:
>
> >  Commit is called
> > after adding each document
> >
> >
> >  you should add enough documents and then calling a commit. commit is a
> > cost operation.
> >  if you want to get latest feeded documents, you could use NRT
> >
> > On Tue, Feb 14, 2012 at 12:47 AM, Huy Le <huyle@springpartners.com>
> wrote:
> >
> > > Hi,
> > >
> > > I am using solr 3.5.  I seeing solr keeps creating new segment files
> > (<1MB
> > > files) so often that it triggers segment merge about every one minute.
> I
> > > search the news archive, but could not find any info on this issue.  I
> am
> > > indexing about 10 docs of less 2KB each every second.  Commit is called
> > > after adding each document. Relevant config params are:
> > >
> > > <mergeFactor>10</mergeFactor>
> > > <ramBufferSizeMB>1024</ramBufferSizeMB>
> > > <maxMergeDocs>2147483647</maxMergeDocs>
> > >
> > > What might be triggering this frequent new segment files creation?
> >  Thanks!
> > >
> > > Huy
> > >
> > > --
> > > Huy Le
> > > Spring Partners, Inc.
> > > http://springpadit.com
> > >
> >
>
>
>
> --
> Huy Le
> Spring Partners, Inc.
> http://springpadit.com
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message