lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aravinth thangasami <aravinththangas...@gmail.com>
Subject Improving Performance by Combining Multiple Fields into Single Field
Date Wed, 21 Jun 2017 08:44:03 GMT
Hi all,

We are doing experiment, that combining multiple fields into single field
as using it as StoredField
While retrieving, Instead of retrieving multiple time, we can do with the
Single call.
we thought of avoiding multiple disk calls for reading multiple fields.

we have an index with million documents and each document having 1000
Fields.
we are using Lucene 4.10

Initially, we combined formed JSON object with multiple key values,
converted to String and stored as a single storedField.
we found that converting JSON to String and getting JSON back from String
consumes more time than the individual reading of fields

further, Instead of using JSON, we thought of using HashMap, so that we can
serialise and store as StoredField. While retrieving we deserialize to get
back  HashMap.

Still, we could get any improvement compared to retrieving as multiple
individual single field.


Please clarify me for these questions.

How bytebuffer loaded for stored fields in a  large document with 1000
fields?
Is there any size limitation for loading bytebuffer?

When the mapping of storedFields happen?
Does it happen during indexsearcher opening?

Is there any better approach to achieve what we are doing?




Thanks
Aravinth

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message