lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <>
Subject [jira] [Updated] (SOLR-1725) Script based UpdateRequestProcessorFactory
Date Sat, 30 Jun 2012 01:52:44 GMT


Hoss Man updated SOLR-1725:

    Attachment: SOLR-1725.patch

Updated patch, changes...

* refactor TestBadConfig and BadIndexSchemaTest to use common base class
* add TestBadConfig tests that bad script processor configs should fail on init
* add initEngine call to inform so errors are triggered at startup
* tweak SolrCore init error reporting to make the underlying msg more visible
* tweak script processor error reporting to have better error msgs
* test wrapping of errors thrown by the script functions
* explicitly check that ScriptEngine is Invocable
* make tests use assume() on existence of javascript engine since JVMs aren't garunteed to
support it
* make processor fail hard if a method is missing from a script
* revert unneccessary SolrResourceLoader change
* fixed rmuirs charset suggestions

...the only thing from my previous list i didn't get to today was polishing up the javadocs
(still some TODOs in there about scoping)


FWIW: Erik and i had a (VERY) breif exchange on IRC last night about the "NoSuchMethod" problem
in which he pointed me to his earlier comment about a simpler syntax for just dealing with
"processAdd" as it's own script, which got me reading a lot of the older comments on this
issue (i fully admit: i glossed over most of them when i first started looking at this issue
a few days ago because when skimming it looked like they were mostly all about Java5 stuff)
which got me thinking more about scoping and caching of ScriptEngine objects.

In general, i think what we have here is definitely usable (if nothing else then for rapid
prototyping and experimentation), and provides a fairly direct representation of an "UpdateProcessor
implemented as a script - ie: isolated instances are constructed for each request (with no
risk of multiple requests polluting each other) and individual functions/methods are called
for each command in that request.  

But there are probably other approaches we could take, in slightly different directions, that
deviate from the existing UpdateProcessor model to get other gains:
* Construct ScriptEngine instances that are scoped to the processorfactory (and live for hte
lifespan of hte solr core), and using eval or invokeFunction for the individual methods passing
req,rsp,and cmd as args (requires the user to be more careful about variable usage in their
script implementations)
* let the user specify distinct script files for each method they want to implement and require
that the ScriptEngines are "Compilable", so we can generate a CompiledScript during processorfactory
initialization, such that the CompiledScript's lifespan is the same sa the solr core, and
call CompiledScript.eval(Bindings) with Bindings containing the req/rsp/cmd in each process
method -- ie: the entire script file is a single function (should be much more efficient,
but has a lot of preconditions, and like the above suggestion requires the user to be careful
about the variables in their scripts)
* etc...

...the bottom line being i think we should commit this, and iterate in other issues on other
ways scripting could be exposed as update processors.

with that in mind, the one pro-active thing i would think we should still do here, is to change
the name to something like "StatelessScriptUpdateProcessorFactory" since the scripts are re-evaluated
from scratch on every request.  that will help us differnetiate it from any alternative future
factory implementations where we try to support compiling/cache/reusing scripts.


> Script based UpdateRequestProcessorFactory
> ------------------------------------------
>                 Key: SOLR-1725
>                 URL:
>             Project: Solr
>          Issue Type: New Feature
>          Components: update
>    Affects Versions: 1.4
>            Reporter: Uri Boness
>            Assignee: Erik Hatcher
>              Labels: UpdateProcessor
>             Fix For: 4.1
>         Attachments: SOLR-1725-rev1.patch, SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch,
SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch,
SOLR-1725.patch, SOLR-1725.patch
> A script based UpdateRequestProcessorFactory (Uses JDK6 script engine support). The main
goal of this plugin is to be able to configure/write update processors without the need to
write and package Java code.
> The update request processor factory enables writing update processors in scripts located
in {{solr.solr.home}} directory. The functory accepts one (mandatory) configuration parameter
named {{scripts}} which accepts a comma-separated list of file names. It will look for these
files under the {{conf}} directory in solr home. When multiple scripts are defined, their
execution order is defined by the lexicographical order of the script file name (so {{scriptA.js}}
will be executed before {{scriptB.js}}).
> The script language is resolved based on the script file extension (that is, a *.js files
will be treated as a JavaScript script), therefore an extension is mandatory.
> Each script file is expected to have one or more methods with the same signature as the
methods in the {{UpdateRequestProcessor}} interface. It is *not* required to define all methods,
only those hat are required by the processing logic.
> The following variables are define as global variables for each script:
>  * {{req}} - The SolrQueryRequest
>  * {{rsp}}- The SolrQueryResponse
>  * {{logger}} - A logger that can be used for logging purposes in the script

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message