lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Toke Eskildsen ...@statsbiblioteket.dk>
Subject Changing the subject for a JIRA-issue (Was: [jira] Created: (LUCENE-2335) optimization: when sorting by field, if index has one segment and field values are not needed, do not load String[] into field cache)
Date Tue, 06 Apr 2010 09:26:23 GMT
The current subject and description of
https://issues.apache.org/jira/browse/LUCENE-2335
is obsolete due to new knowledge.

Is it possible to change it? If not, what is the policy here? To open a
new issue and close the old one?

Cc: To Michael McCandless as he is the reporter of the issue.


If it can be changed, I would like to propose the following:

Optimization: Locale-based sort by field with low memory overhead

The current implementation of locale-based sort in Lucene uses the
FieldCache which keeps all sort-terms in memory. Beside the huge memory
overhead, searching requires comparison of terms with collator.compare
every time, making searches with millions of hits fairly expensive.

An idea for an alternative implementation is to create a packed list of
pre-sorted ordinals for the sort terms and a map from document-IDs to
entries in the sorted ordinals list.

This results in very low memory overhead and faster sorted searches, at
the cost of increased startup-time. As the ordinals can be resolved to
terms after the sorting has been performed, this approach supports
fillFields=true.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org


Mime
View raw message