lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SOLR-1725) Script based UpdateRequestProcessorFactory
Date Thu, 28 Jun 2012 01:06:44 GMT

    [ https://issues.apache.org/jira/browse/SOLR-1725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13402743#comment-13402743
] 

Hoss Man commented on SOLR-1725:
--------------------------------

bq. My comment above still stands: "Another TODO is to get this to work with a scripting language
implementation JAR file being added as a "plugin" somehow.

I played around with this on the train today and confirmed that we can do runtime loading
of jars that included script engines if we changed the ScriptEngineManager instantiation so
that we use the one arg constructor and pass in resourceLoader.getClassLoader().

A few other notes based on reviewing the patch and playing arround with it.  Baring objections
i'll probably take a stab at addressing these tomorow or friday...

* i don't see any mechanism for scripts to indicate that processing should "stop" -- ie: the
way a java UpdateProcessor would just return w/o calling super.foo. we should add/test/doc
some functionality to look at the result of the invokeFunction call to support this
* the tests seem to assert that the side effects of the scripts happen (ie: that the "testcase"
records the function names) but i don't see any assertions that the expected modifications
of the update commands is happening (ie: that documents are being modified in processAdd
* we need to test that request params are easily accessable (i'm not sure how well the SolrQueryRequest
class works in various scripting langauges, so might need to pull out hte SolrParams and expose
directly - either way we need to demonstrate doing it in a test)
* whitespace/comma/pipesplitting of the script names is a bad meme.  we should stop doing
that, and require that multiple scripts be specified as multiple {{<str>}} params
** we can add convenience code to support {{<arr name="script"><str><str></arr>}}
style as well
* "ScriptFile" and it's extension parsing is very primitive and broken on any file with "."
in it's name.  We should just use the helper method for parsing filename extensions that already
exists in commons-io
* from what i can tell looking at the ScriptEngine javadocs, it's possible that a ScriptEngine
might exist w/o a specific file extension, or that multiple engines could support the same
extension(s)  we should offer an init param that lets the user specify a ScriptEngine by "shortname"
to override whatever extension might be found
* currently, problems with scripts not being found, or engines for scripts not being found,
aren't reported until first request tries to use them - we should error check all of this
in init (or inform) and fail fast.
** ditto for the assumption in invokeFunction that we can cast every ScriptEngine to Invocable
-- we should at check this on init/inform and fail fast
* the way the various UpdateProcessor methods are implemented to be lenient about any scripts
that don't explicitly implement a method seems kludgy -- isn't there anyway we can introspect
the engine to ask if a function exists?
** in particular, when i did some testing with jruby, i found that it didn't work at all -
i guess jruby was throwing a ScriptException instead of NoSuchMethodException?

{noformat}
undefined method `processCommit' for main:Object (NoMethodError)
org.jruby.embed.InvokeFailedException: (NoMethodError) undefined method `processCommit' for
main:Object
	at org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.call(EmbedRubyObjectAdapterImpl.java:403)
	at org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.callMethod(EmbedRubyObjectAdapterImpl.java:189)
	at org.jruby.embed.ScriptingContainer.callMethod(ScriptingContainer.java:1386)
	at org.jruby.embed.jsr223.JRubyEngine.invokeFunction(JRubyEngine.java:262)
	at org.apache.solr.update.processor.ScriptUpdateProcessorFactory$ScriptUpdateProcessor.invokeFunction(ScriptUpdateProcessorFactory.java:221)
	at org.apache.solr.update.processor.ScriptUpdateProcessorFactory$ScriptUpdateProcessor.processCommit(ScriptUpdateProcessorFactory.java:202)
{noformat}



                
> Script based UpdateRequestProcessorFactory
> ------------------------------------------
>
>                 Key: SOLR-1725
>                 URL: https://issues.apache.org/jira/browse/SOLR-1725
>             Project: Solr
>          Issue Type: New Feature
>          Components: update
>    Affects Versions: 1.4
>            Reporter: Uri Boness
>            Assignee: Erik Hatcher
>              Labels: UpdateProcessor
>             Fix For: 4.1
>
>         Attachments: SOLR-1725-rev1.patch, SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch,
SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch, SOLR-1725.patch
>
>
> A script based UpdateRequestProcessorFactory (Uses JDK6 script engine support). The main
goal of this plugin is to be able to configure/write update processors without the need to
write and package Java code.
> The update request processor factory enables writing update processors in scripts located
in {{solr.solr.home}} directory. The functory accepts one (mandatory) configuration parameter
named {{scripts}} which accepts a comma-separated list of file names. It will look for these
files under the {{conf}} directory in solr home. When multiple scripts are defined, their
execution order is defined by the lexicographical order of the script file name (so {{scriptA.js}}
will be executed before {{scriptB.js}}).
> The script language is resolved based on the script file extension (that is, a *.js files
will be treated as a JavaScript script), therefore an extension is mandatory.
> Each script file is expected to have one or more methods with the same signature as the
methods in the {{UpdateRequestProcessor}} interface. It is *not* required to define all methods,
only those hat are required by the processing logic.
> The following variables are define as global variables for each script:
>  * {{req}} - The SolrQueryRequest
>  * {{rsp}}- The SolrQueryResponse
>  * {{logger}} - A logger that can be used for logging purposes in the script

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message