lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Hostetter <hossman_luc...@fucit.org>
Subject Re: Status: Sorting on tokenized fields
Date Tue, 26 Sep 2006 00:24:20 GMT

: In our application we need to do this for all 20 fields. That means
: me have to create twenty redundant fields just for sorting.
: That's really an overhead in size and indexing-time.

I guess it just depends on the size of your index and how fast is "fast
enough" when indexing ... most people are more concerned abotu the query
speed then the index time speed, and doing a concatenation at query time
when building the FieldCache every time a new IndexReader is opened is
probably a less optimal choice then doing the concat once at index time.

: :: using the stored value doesn't help: there can be multiple stored values
: :: just as easily as there can be multiple tokens.
:
: In what situations do you have more than one stored value per field?

oh, all the time ... consider the case of multiple authors for a book --
"Erik Hatcher" and "Otis Gospodnetic" ... it makes perfect sense to both
index/store both author names (probably using a large position gap in your
analyzer) but now your "author" field isn't sortable, so you make an
untokenized string version that has the specific sort string that you want
which isn't neccessarily the same as the concated values (perhaps:
"Hatcher, Erik & Gospodnetic, Otis")

: In our applications we always have one stored value for each field.
: In such a situation it would be perfect to take these stored values for
: sorting....
: Wouldn't it?

Now that there is Lazy Field loading, people have discussed the prospect
of SortComparatorSources using stored fields to do sorting without needing
to build FieldCache's at all -- perhaps you want to look into that option?

Alternately: if your index changes very infrequently, you could forcibly
populate the FieldCache with any data you want, from any source you want
-- using Stored fields, or concatenating tokens.



-Hoss


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message