lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shai Erera <ser...@gmail.com>
Subject Extending Sort/FieldCache
Date Thu, 20 Aug 2009 11:49:54 GMT
Hi

I'd like to extend Lucene's FieldCache such that it will read native values
from a different place (in my case, payloads). That is, instead of iterating
on a field's terms and parsing each String to long (for example), I'd like
to iterate over one term (sort:long, again - an example) and decode each
payload value to long, and store it in the cache. The reason I want to
extend Lucene's FieldCache is because I'd like Lucene to take care of
updating this cache when necessary (such as after reopen for example). This
will allow me to use Lucene's Sort option more easily.

I noticed Sort can be extended by providing a CUSTOM SortField, but that
forces me to create Comparable objects, which is much more expensive than
native longs (40 bytes vs. 8 bytes).

I didn't find a way though to extend FieldCache, or ExtendedFieldCache -->
even though both are extendable, I don't find the place where they're given
as input to TopFieldDocCollector, FieldSortedHitQueue etc. Perhaps I'm
missing it?

I need to do it on top of 2.4.1, but if it's not possible and something was
done in 2.9 (which I missed) in that regard I'd be happy to get a pointer to
that.

Today, I use this payload to implement sorting, which is memory efficient
for very large indexes, but for small indexes I think Lucene's built-in sort
will perform better.

BTW, if it interests anyone, perhaps augmenting Lucene's sort w/ reading
values from a Payload, or doing a complete payload-based-sort, I can work up
a patch ...

Thanks
Shai

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message