lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aravinth thangasami <aravinththangas...@gmail.com>
Subject Re: Improving Performance by Combining Multiple Fields into Single Field
Date Wed, 21 Jun 2017 16:48:03 GMT
Thanks Adrien

On Wed, Jun 21, 2017 at 5:33 PM, Adrien Grand <jpountz@gmail.com> wrote:

> The file is mapped when the index reader is open. Retrieving one or more
> fields always requires a single disk seek since all values for a given
> document are store together, just make sure to perform a single call to
> IndexReader.document with the list of fields that you want to retrieve
> rather than call this method once for each field, or you will pay the price
> for decompression every time.
>
> Le mer. 21 juin 2017 à 10:44, aravinth thangasami <
> aravinththangasami@gmail.com> a écrit :
>
> > Hi all,
> >
> > We are doing experiment, that combining multiple fields into single field
> > as using it as StoredField
> > While retrieving, Instead of retrieving multiple time, we can do with the
> > Single call.
> > we thought of avoiding multiple disk calls for reading multiple fields.
> >
> > we have an index with million documents and each document having 1000
> > Fields.
> > we are using Lucene 4.10
> >
> > Initially, we combined formed JSON object with multiple key values,
> > converted to String and stored as a single storedField.
> > we found that converting JSON to String and getting JSON back from String
> > consumes more time than the individual reading of fields
> >
> > further, Instead of using JSON, we thought of using HashMap, so that we
> can
> > serialise and store as StoredField. While retrieving we deserialize to
> get
> > back  HashMap.
> >
> > Still, we could get any improvement compared to retrieving as multiple
> > individual single field.
> >
> >
> > Please clarify me for these questions.
> >
> > How bytebuffer loaded for stored fields in a  large document with 1000
> > fields?
> > Is there any size limitation for loading bytebuffer?
> >
> > When the mapping of storedFields happen?
> > Does it happen during indexsearcher opening?
> >
> > Is there any better approach to achieve what we are doing?
> >
> >
> >
> >
> > Thanks
> > Aravinth
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message