lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: indexing performance 6.6 vs 7.1
Date Thu, 18 Jan 2018 18:03:30 GMT
Robert:

Ah, right. I keep confusing my gmail lists
"lucene dev"
and
"lucene list"....

Siiigggghhhhh.



On Thu, Jan 18, 2018 at 9:18 AM, Adrien Grand <jpountz@gmail.com> wrote:
> If you have sparse data, I would have expected index time to *decrease*,
> not increase.
>
> Can you enable the IW info stream and share flush + merge times to see
> where indexing time goes?
>
> If you can run with a profiler, this might also give useful information.
>
> Le jeu. 18 janv. 2018 à 11:23, Rob Audenaerde <rob.audenaerde@gmail.com> a
> écrit :
>
>> Hi all,
>>
>> We recently upgraded from Lucene 6.6 to 7.1.  We see a significant drop in
>> indexing performace.
>>
>> We have a-typical use of Lucene, as we (also) index some database tables
>> and add all the values as AssociatedFacetFields as well. This allows us to
>> create pivot tables on search results really fast.
>>
>> These tables have some overlapping columns, but also disjoint ones.
>>
>> We anticipated a decrease in index size because of the sparse docvalues. We
>> see this happening, with decreases to ~50%-80% of the original index size.
>> But we did not expect an drop in indexing performance (client systems
>> indexing time increased with +50% to +250%).
>>
>> (Our indexing-speed used to be mainly bound by the speed the Taxonomy could
>> deliver new ordinals for new values, currently we are investigating if this
>> is still the case, will report later when a profiler run has been done)
>>
>> Does anyone know if this increase in indexing time is to be expected as
>> result of the sparse docvalues change?
>>
>> Kind regards,
>>
>> Rob Audenaerde
>>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message