lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joel Bernstein <joels...@gmail.com>
Subject Re: In-memory docValues speedup
Date Wed, 22 Jun 2016 10:57:56 GMT
I've tested with the Direct docValuesFormat which is uncompressed
in-memory. But I haven't seen any noticeable performance gain. I've been
meaning to dig into exactly why I wasn't seeing a performance gain, but
haven't had the chance to do this yet.

If you test out the Direct docValuesFormant, I'd be interested in hearing
your findings.



Joel Bernstein
http://joelsolr.blogspot.com/

On Wed, Jun 22, 2016 at 5:37 AM, Aashish Dattani <aashish@zettata.com>
wrote:

> Hi everyone,
>
> I have 2 million documents in my Solr index. I have enabled docValues on
> one
> of the integer fields, and set its docValuesFormat to Memory. This is
> because I want to have very quick forward lookups on this field in my
> custom
> component.
>
> I am running my Solr installation on a 35GB RAM machine, so memory is not a
> big resource constraint.
>
> I was expecting that making the docValuesFormat to Memory will make the
> docValues access to be as fast as a simple array lookup - but it didn't.
> Upon digging deeper and inspecting the MemoryDocValuesProducer class, it
> seems like Lucene is doing a delta compression and storing it in a
> PackedInts datastructure, which is making the lookups slower.
>
> Is there any configuration/implementation that I can use to make sure that
> the docValues are faster than they are right now - essentially any way to
> ensure that it is stored as an in-memory array? Do you see any downsides to
> using this approach?
>
> I'd appreciate any help on this.
>
> Thanks!
>
>
>
> --
> View this message in context:
> http://lucene.472066.n3.nabble.com/In-memory-docValues-speedup-tp4283783.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message