lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eric Torti <ericzu...@gmail.com>
Subject Re: Is solr.StandardDirectoryFactory an MMapDirectory?
Date Fri, 09 Oct 2015 16:14:19 GMT
Ok, thanks Shawn!

That makes sense. We'll be experimenting with it.

Best,
Eric

On Wed, Oct 7, 2015 at 5:54 PM, Shawn Heisey <apache@elyograg.org> wrote:
> On 10/7/2015 12:00 PM, Eric Torti wrote:
>> Can we read "high reopen rate" as "frequent soft commits"? (In our
>> case, hard commits do not open a searcher. But soft commits do).
>>
>> Considering it does mean "frequent soft commits", I'd say that it
>> doesn't fit our setup because we have an index rate of about 10
>> updates/s and we perform a soft commit at each 15min. So our scenario
>> is not near real time in that sense. In light of this, do you thing
>> using NRTCachingDirectory is still convenient?
>
> The NRT factory achieves high speed in NRT situations by flushing very
> small updates to RAM instead of the disk.  As more updates come in,
> older index segments sitting in RAM will eventually be flushed to disk,
> so a sustained flood of updates doesn't really achieve a speed increase,
> but a short burst of updates will be searchable *very* quickly.
>
> NRTCachingDirectoryFactory was chosen for Solr examples (and I think
> it's the Solr default) because it has no real performance downsides, but
> has a strong possibility to be noticeably faster than the standard
> factory in NRT situations.
>
> The only problem with it is that small index segments from recent
> updates might only exist in RAM, and not get flushed to disk, so they
> would be lost if Solr dies or is killed suddenly.  This is part of why
> the updateLog feature exists -- when Solr is started, the transaction
> logs will be replayed, inserting/replacing (at a minimum) all documents
> indexed since the last hard commit.  When the replay is finished, you
> will not lose data.  This does require a defined uniqueKey to operate
> correctly.
>
> Thanks,
> Shawn
>

Mime
View raw message