lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrien Grand <jpou...@gmail.com>
Subject Re: Recommended number of fields in one lucene index
Date Wed, 15 Feb 2017 19:08:12 GMT
I think it is hard to come up with a general rule, but there is certainly a
per-field overhead. There are some things that we need to store per field
per segment in memory, so if you multiply the number of fields you have,
you could run out of memory. In most cases I have seen where the index had
so many fields, it was due to the fact that the application wanted to index
arbitrary documents and provide search for them, which cannot scale, or to
the fact that the index contained many unrelated documents that should have
been put into different indices. This limit has been very useful to catch
such design problems early instead of waiting for the production server to
go out of memory due to the multiplication of fields.

Le mer. 15 févr. 2017 à 19:44, Kumaran Ramasubramanian <kums.134@gmail.com>
a écrit :

> While searching, i use _all_ blob field to search in texts of all fields
> data.
>

This is interesting: if all your searches go to a catch-all field, then it
means that you do not need those thousands of fields but could just have a
single indexed field that is used for searching, and a binary blob that
stores all the data so that you can perform updates. So this only requires
two fields from a Lucene perspective.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message