lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aashish Dattani <aash...@zettata.com>
Subject In-memory docValues speedup
Date Wed, 22 Jun 2016 09:37:35 GMT
Hi everyone,

I have 2 million documents in my Solr index. I have enabled docValues on one
of the integer fields, and set its docValuesFormat to Memory. This is
because I want to have very quick forward lookups on this field in my custom
component. 

I am running my Solr installation on a 35GB RAM machine, so memory is not a
big resource constraint.

I was expecting that making the docValuesFormat to Memory will make the
docValues access to be as fast as a simple array lookup - but it didn't.
Upon digging deeper and inspecting the MemoryDocValuesProducer class, it
seems like Lucene is doing a delta compression and storing it in a
PackedInts datastructure, which is making the lookups slower. 

Is there any configuration/implementation that I can use to make sure that
the docValues are faster than they are right now - essentially any way to
ensure that it is stored as an in-memory array? Do you see any downsides to
using this approach?

I'd appreciate any help on this.

Thanks!



--
View this message in context: http://lucene.472066.n3.nabble.com/In-memory-docValues-speedup-tp4283783.html
Sent from the Solr - User mailing list archive at Nabble.com.

Mime
View raw message