lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ryan McKinley (Commented) (JIRA)" <>
Subject [jira] [Commented] (SOLR-2842) Re-factor UpdateChain and UpdateProcessor interfaces
Date Mon, 17 Oct 2011 17:42:10 GMT


Ryan McKinley commented on SOLR-2842:

rather then use UpdateProcessor directly, what about adding a simple interface like:
 SolrInputDocument transform(SolrInputDocument)
and using simple bean getter/setters -- perhaps also respecting the 'aware' interfaces (SolrCoreAware,
SchemaAware, ResourceLoaderAware)

It seems like most of the custom things we would want to do only care about 'add' and don't
care about commit,delete,merge,rollback.  Starting with a simple interface like this would
give us lots of flexibility to integrate wherever it feels most appropriate -- client/server
or any other pipeline framework (I've been using commons pipeline with pretty reasonable success)

> Re-factor UpdateChain and UpdateProcessor interfaces
> ----------------------------------------------------
>                 Key: SOLR-2842
>                 URL:
>             Project: Solr
>          Issue Type: Improvement
>          Components: update
>            Reporter: Jan H√łydahl
> The UpdateChain's main task is to send SolrInputDocuments through a chain of UpdateRequestProcessors
in order to transform them in some way and then (typically) indexing them.
> This generic "pipeline" concept would also be useful on the client side (SolrJ), so that
we could choose to do parts or all of the processing on the client. The most prominent use
case is extracting text (Tika) from large binary documents, residing on local storage on the
client(s). Streaming hundreds of Mb over to Solr for processing is not efficcient. See SOLR-1526.
> We're already implementing Tika as an UpdateProcessor in SOLR-1763, and what would be
more natural than reusing this - and any other processor - on the client side?
> However, for this to be possible, some interfaces need to change slightly..

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message