groovy-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Korbee Reinout <Reinout.Kor...@snb.ch>
Subject RE: Executing Groovy DSL scripts concurrently
Date Tue, 13 Feb 2018 11:01:46 GMT
I use the Groovy Script as a sort of function that returns a specific type and call it repeatedly
with different bindings, the following is an excerpt of my code:

private static final GroovyClassLoader GROOVY_CLASSLOADER = new GroovyClassLoader();
private static final Map<String, Script> functionCache = new HashMap<>();

public static MyReturnType evaluate(String functionString, Object myDataToBind) {

Script script = null;
          try {
                script = resolveFunction(functionString);
          } catch (Exception e)  {
                throw new IllegalArgumentException("The function could not be resolved: "
+ functionString, e);
          }
          script.getBinding().setVariable(VAR, myDataToBind);
          Object o = null;
          try {
                o = script.run();
          } catch (Exception e) {
                throw new IllegalArgumentException("The function is not valid: " + functionString,
e);
          }
          MyReturnType ts = null;
          if(o instanceof MyReturnType) {
                ts = (MyReturnType) o;
          } else {
                throw new IllegalArgumentException("The function doesn't evaluate to a MyReturnType:
" + functionString);
          }

          return ts;

}


Script resolveFunction(String f) throws InstantiationException, IllegalAccessException  {
if(functionCache.containsKey(f)) {    // functions are likely repeated many times in one chunk
     return functionCache.get(f);
}

     // fiddle a bit with the string representation of the script

Class<?> clazz = GROOVY_CLASSLOADER.parseClass(f);
Object cc = clazz.newInstance();
Script script = (Script) cc;
functionCache.put(f, script);
return script;
}






From: Ralph Johnson [mailto:rjohnson.uiuc@gmail.com]
Sent: Tuesday, February 13, 2018 3:41 AM
To: users@groovy.apache.org
Subject: Re: Executing Groovy DSL scripts concurrently

That guide to integrating is *very* interesting!   I had never seen GroovyClassLoader before.

I work with a large application that uses GroovyShell a lot.  It has thousands of scripts.
  Sometimes they call each other recursively, which runs into the same problem with shared
Bindings that concurrency has.    I wondered why a script couldn't be a class so that I could
instantiate it each time it ran, and so not have to share bindings.   I am pretty sure that
with GroovyClassLoader, I can do it.    Thanks for pointing it out!

-Ralph Johnson

On Mon, Feb 12, 2018 at 2:10 AM, Korbee Reinout <Reinout.Korbee@snb.ch<mailto:Reinout.Korbee@snb.ch>>
wrote:
[Boxbe]<https://www.boxbe.com/overview>[http://www.boxbe.com/stfopen?tc_serial=36637963201&tc_rand=1084693400&utm_source=stf&utm_medium=email&utm_campaign=ANNO_CLEANUP_ADD&utm_content=001]This
message is eligible for Automatic Cleanup! (Reinout.Korbee@snb.ch<mailto:Reinout.Korbee@snb.ch>)
Add cleanup rule<https://www.boxbe.com/popup?url=https%3A%2F%2Fwww.boxbe.com%2Fcleanup%3Fkey%3DIACL3oJKfUFUI8o%252BV%252BSvhRz7CVhfgVAef%252BSiCMBkkH0%253D%26token%3DmOLRlI1UuZw8SVNtchg7i08qve4GprNHZPJ%252FuFCSHStD%252F7QwDC7CiD1J%252FXpeem8HBaBmBiSO7lwH0XUlOXqrrSd4Lxlv%252BVtlgDCgdh9yWPHzz1U9J3EMThUh%252B8pDNwMq4JUlD72gGHFJICg5jMje9A%253D%253D&tc_serial=36637963201&tc_rand=1084693400&utm_source=stf&utm_medium=email&utm_campaign=ANNO_CLEANUP_ADD&utm_content=001>
| More info<http://blog.boxbe.com/general/boxbe-automatic-cleanup?tc_serial=36637963201&tc_rand=1084693400&utm_source=stf&utm_medium=email&utm_campaign=ANNO_CLEANUP_ADD&utm_content=001>

Hi,

According to: http://docs.groovy-lang.org/latest/html/documentation/guide-integrating.html

“You must be very careful when using shared data in a multithreaded environment. The Binding
instance that you pass to GroovyShell is not thread safe, and shared by all scripts.”

There are some guidelines regarding multi-threading and thread-safety. Before attempting something
like this, ask yourself the question, “can I guarantee that my code is thread safe?”,
which is very different from running a successful test without concurrency issues.

You could use the GroovyClassLoader directly and keep the compiled scripts in a cache, see
the mentioned docs on how to obtain thread-safety with the Binding object.

From: K Adithyan (tech) [mailto:adi.k.dev@gmail.com<mailto:adi.k.dev@gmail.com>]
Sent: Wednesday, February 7, 2018 4:15 PM
To: users@groovy.apache.org<mailto:users@groovy.apache.org>
Subject: Executing Groovy DSL scripts concurrently

Team,

Our application is a Root Cause Analysis Computation Engine for Telecom networks. We have
been implementing the logics in java only so far.

We are now planning to write DSLs for each feature and write the logics using the DSL. We
have implemented a framework for plugging in any number of DSLs and integrated that DSL Framework
with our application.

Each of the logics written over the DSL are executed concurrently using 40 or 50 or 60 threads
based on the rate of the incoming events. Due to the concurrency, we are now creating `GroovyShell`
object for every execution and calling `shell.evaluate(ourScriptFile)` to run the DSL scripts
with our own Binding object and delegates.

This prevents the system to run the logic at its own speed. Everytime parsing and running
takes significant time which reduces the event handling capacity of the application.

I have attempted with `GroovyShell` and `GroovyScriptEngine`. But both are offering same performance
levels. GroovyShell is slightly better than GroovyScriptEngine.

Under these circumstances, what is the best way to wrap our script logic, that is based on
our own DSLs, and run it concurrently without any overhead for parsing, compilation, etc?????????


Pls advice

Reg,
Adithyan K

This e-mail message contains confidential information and is intended only for the named recipient(s).
If you are not an intended recipient, any disclosure, copying or distribution is prohibited.
Please notify the sender immediately by e-mail if you have received this message in error
and delete this message from your system. As internet communications are not secure, the Swiss
National Bank accepts no liability for any errors or omissions in the contents of this message.
Any views expressed in this message are those of the individual sender, except where the sender
specifically states them to be the views of the Swiss National Bank.


Mime
View raw message