lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: segment size buffersize vs commit time & Tlog
Date Wed, 06 Mar 2013 00:16:10 GMT
1) that's pretty much it. But the number of segments will change
    all the time based on your merge policy. Great visualizatin
    of segment merging from Mike McCandless here:
    http://www.youtube.com/watch?v=YOklKW9LJNY

2) What do you mean a "failure with the tlog"? Unfortunately
    there's not a lot of great documentation. The short form
    is that the documents are written to the tlog between hard
    commits (openSearcher true/false irrelevant). Enough
    docs are kept to recover at least 100 documents. If the failed
    node needs more than 100 docs, an old-style replication is
    done from the leader and things pending in the leader are
    then sent to the recovering node, at which point things
    return to normal.

Best
Erick


On Tue, Mar 5, 2013 at 7:53 AM, Manuel Le Normand <
manuel.lenormand@gmail.com> wrote:

> 1) While indexing documents to a solr shard, I do not seem to understand
> how to segments get formed? Is it only dependent of the maxBufferSize
> attribute or also the (hard) commit time? Because when i hard commit all
> the buffer should be written to the disk, right?
>
> Is it simply the minSize(maxBufferSize, bufferSize until i made commit)? In
> a case of low indexing rate I would have a big amount of very small
> segments (ignoring for instance the merging activity) each created every
> hard commit (~2 hours)?
>
> 2) where can i find good info for how a server recovers after failure with
> the tlog? Couldn't find a good one
> Thanks
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message