lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From P Williams <williams.tricia.l...@gmail.com>
Subject Re: Changing Cache Properties after Indexing
Date Fri, 17 Jan 2014 16:31:50 GMT
You're both completely right.  There isn't any issue with indexing with
large cache settings.

I ran the same indexing job five times, twice with large cache and twice
with the default values. I threw out the first job because no matter if
it's cached or uncached it runs ~2x slower. This must have been the
observation I based my incorrect caching notion on.

I unloaded with delete of the data directory and reloaded the core each
time.  I'm using DIH with the FileEntityProcessor and
PlainTextEnityProcessor to index ~11000 fulltext books.

w/ cache
0:13:14.823
0:12:33.910

w/o cache
0:12:13.186
0:15:56.566

There is variation, but not anything that could be explained by the cache
settings. Doh!

Thanks,
Tricia


On Mon, Jan 13, 2014 at 6:08 PM, Shawn Heisey <solr@elyograg.org> wrote:

> On 1/13/2014 4:44 PM, Erick Erickson wrote:
>
>> On the face of it, it's somewhat unusual to have the cache settings
>> affect indexing performance. What are you seeing and how are you indexing?
>>
>
> I think this is probably an indirect problem.  Cache settings don't
> directly affect indexing speed, but when autoWarm values are high and NRT
> indexing is happening, new searchers are requested frequently and the
> autoWarm makes that happen slowly with a lot of resources consumed.
>
> Thanks,
> Shawn
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message