lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joel Bernstein (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (SOLR-4465) Configurable Collectors
Date Tue, 12 Mar 2013 20:23:15 GMT

    [ https://issues.apache.org/jira/browse/SOLR-4465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13600410#comment-13600410
] 

Joel Bernstein edited comment on SOLR-4465 at 3/12/13 8:21 PM:
---------------------------------------------------------------

Added support for delegating collectors.

This design allows for a "topdocs" collector to be wrapped by "delegating" collectors.

The "topdocs" collector collects the doclist and docset. The delegating collectors are designed
to collect aggregate data of some kind.

Now there are two collector parameters:

cl.topdocs=<topdocs collector name>
cl.delegating=<comma separated list of delegating collectors>

Both of these parameters refer to collectorFactories configured in the solrconfig.xml.

Parameters are passed to the collectors by name. For example:

cl.topdocs=default&cl.delegating=sum&cl.sum.groupby=field1&cl.sum.column=field2

In this example the topdocs collector is the "default" collector. The delegating collector
is the "sum" collector. Both of these refer to named collectorFactories in solrconfig.xml.
The sum collector is being passed two parameters "groupby" and "column", telling it to groupby
field1 and sum field2. 


The delegating collectors have access to the ResponseBuilder and through that can add Maps
directly to the SolrQueryResponse.

Both the topdocs collector and the delegating collectors take part in the merge of distributed
results from shards.

This paves the way for pluggable distributed analytics to be included with searches results.

TODO: I believe Maps that are placed in the SolrQueryResponse are automatically output but
some work needs to be done get them read in the solrj QueryResponse class so they can be merged.

A simple example delegating collector to test the entire flow needs to be created.
                
      was (Author: joel.bernstein):
    Added support for delegating collectors.

This design allows for a "topdocs" collector to be wrapped by "delegating" collectors.

The "topdocs" collector collects the doclist and docset. The delegating collectors are designed
to collect aggregate data of some kind.

Now there are two collector parameters:

cl.topdocs=<topdocs collector name>
cl.delegating=<comma separated list of delegating collectors>

Both of these parameters refer to collectorFactories configured in the solrconfig.xml.

Parameters are passed to the collectors by name. For example:

cl.topdocs=default&cl.delegating=sum&cl.sum.groupby=field1&cl.sum.column=field2

In this example the topdocs collector is the default. The delegating collector is the "sum"
collector. The sum collector is being passed two parameters "groupby" and "column", telling
it to groupby field1 and sum field2. 


The delegating collectors have access to the ResponseBuilder and through that can add Maps
directly to the SolrQueryResponse.

Both the topdocs collector and the delegating collectors take part in the merge of distributed
results from shards.

This paves the way for pluggable distributed analytics to be included with searches results.

TODO: I believe Maps that are placed in the SolrQueryResponse are automatically output but
some work needs to be done get them read in the solrj QueryResponse class so they can be merged.

A simple example delegating collector to test the entire flow needs to be created.
                  
> Configurable Collectors
> -----------------------
>
>                 Key: SOLR-4465
>                 URL: https://issues.apache.org/jira/browse/SOLR-4465
>             Project: Solr
>          Issue Type: New Feature
>          Components: search
>    Affects Versions: 4.1
>            Reporter: Joel Bernstein
>             Fix For: 4.3
>
>         Attachments: SOLR-4465.patch, SOLR-4465.patch, SOLR-4465.patch, SOLR-4465.patch,
SOLR-4465.patch, SOLR-4465.patch, SOLR-4465.patch, SOLR-4465.patch, SOLR-4465.patch, SOLR-4465.patch,
SOLR-4465.patch, SOLR-4465.patch
>
>
> This issue is to add configurable custom collectors to Solr. This expands the design
and work done in issue SOLR-1680 to include:
> 1) CollectorFactory configuration in solconfig.xml
> 2) Http parameters to allow clients to dynamically select a CollectorFactory and construct
a custom Collector.
> 3) Make aspects of QueryComponent pluggable so that the output from distributed search
can conform with custom collectors at the shard level.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message