lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless (JIRA)" <>
Subject [jira] [Commented] (LUCENE-2883) Consolidate Solr & Lucene FunctionQuery into modules
Date Thu, 19 May 2011 21:24:48 GMT


Michael McCandless commented on LUCENE-2883:

bq. Hmm good question. This looks to be related to sorting by FQ (SOLR-1297) because some
FQs need to be weighted. Not sure what to do here yet... which FQs in particular require this?

Both all of them and not many of them (complicated). The sorting of FQ
functionality is necessary for all FQs in Solr since the user can sort
by any FQ. However the extension made by the SolrSortField is the ability
to create a 'weighted' SortField by passing in a IndexSearcher and having
it stored in a Map. The Map is then made available to any DocValues when
they create their values.

So the goal here is to make the top-level searcher (IR) visible to the
FQ's getValues?  I think this pre-dated the cutover to
AtomicReaderContext, which now provides the top reader?  Maybe this
isn't needed anymore...?

This is when the 'not many' comes into effect. Only a few DocValues implementations
use the contents of the Map. DocFreqValueSource for example uses the IndexSearcher
in the Map. But I suppose there could be many user implementations that do.

EG DocFreqValueSource could pull docFreq from the top reader?

Though QueryValueSource needs a searcher (but, seems to make one, from
the top reader, if it wasn't provided one).

SolrSortField is currently used in SolrIndexSearcher to 'weight' the Sorts. I wonder
whether we can simplify this? When ValueSource#getSort is called (which is only in 1
place really), we can pass in the IndexSearcher, meaning the SortField can be 'weighted'
then and there.

Since SolrSortField is only used in this 1 place currently, we can then revisit dropping it?

That seems good too?

bq. Do you think its worth opening an issue to address this first?

Yes can you do that, and mark this issue as depending in it?

bq. I think apply 90/10 rule here? Start with the easy-to-move queries? We don't need initial
go to be perfect... progress not perfection.

Could we sort the initial commit out and then I can move them over in batches?
Already have a 108k patch, I'd say moving what we can will push it towards 300k

Good question... maybe we can do this on a branch?

bq. Do you have a sense of whether Solr's FQs are a superset of Lucene's? Ie, is there anything
Lucene's FQs can do that Solr's can't?

Solr FQs are hugely more advanced than the ValueSourceQuery based stuff in Lucene.
Its not a full 1 to 1 change since the APIs are slightly different, but I'd say
that we'd want users to use the FQ line of classes. I cant see anything in Lucene's
VSQs that you couldn't do using FQs.

OK then once we finally have the "superset" moved into the module we
should remove Lucene's (deprecate on 3.x).

> Consolidate Solr  & Lucene FunctionQuery into modules
> -----------------------------------------------------
>                 Key: LUCENE-2883
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Task
>          Components: core/search
>    Affects Versions: 4.0
>            Reporter: Simon Willnauer
>              Labels: gsoc2011, lucene-gsoc-11, mentor
>             Fix For: 4.0
>         Attachments: LUCENE-2883.patch
> Spin-off from the [dev list |]

This message is automatically generated by JIRA.
For more information on JIRA, see:

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message