groovy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alain Stalder <astal...@span.ch>
Subject Groovy Grape/Ivy thread safe?
Date Sun, 26 Apr 2015 15:52:49 GMT
Hi there,

In the Java unit test listed further below, I create 100 GroovyShell 
instances and add the same directory to the classpath of the 
GroovyClassLoader of each GroovyShell. This directory contains 100 
script files, named Util0.groovy to Util99.groovy, containing the 
following script (with XX replaced by 0..99):

@Grab('com.google.guava:guava:18.0')
import com.google.common.base.Ascii
class UtilXX {
   static boolean isUpperCase(def c) {
     return Ascii.isUpperCase(c)\n"
   }
}

The unit test then runs

     shell.evaluate("return Util" + j + ".isUpperCase('C' as char)");

in 100 separate threads (with j from 0..99).

This test does not always fail, but often. Most of the time, a 
ConcurrentModificationException occurs down in ivy (which is used by 
Grape). In other cases, a MultipleCompilationErrorsException ocurred at 
"@Grab('com.google.guava:guava:18.0')". (Full stacktraces at the bottom.)

It looks to me like Grape resp. ivy is not thread safe.

Question 1: Is that a bug or just how things currently are? (If it's a 
bug, rather Grape bug or an ivy bug, resp. where would I best file it?)

Question 2: If it's not a bug, is that a general issue, i.e. if I 
compile two sets of sources with the GroovyCompiler (say I create a 
CompilationUnit instance and add sources) will there be similar issues 
if both sets of sources have the same Grape dependencies? (Yes, I know, 
I could just try, but maybe the answer obvious to someone who knows the 
implementation.)

(I have tested this with Groovy 2.4.3 and ivy 2.4.0 on MacOS X with Java 
6, but also I have also seen the same issues on CentOS with several 
Groovy/ivy versions and Java 7, on a deployed webapp, so it seems to be 
fairly independent of the exact environment.)

Best wishes,

Alain


-------------------------------------
Java Unit Test (does not always fail)
-------------------------------------

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertTrue;
import groovy.lang.GroovyShell;

import java.io.File;
import java.io.FileNotFoundException;
import java.io.PrintWriter;
import java.io.UnsupportedEncodingException;
import java.util.LinkedList;
import java.util.List;

import org.junit.Before;
import org.junit.Rule;
import org.junit.Test;
import org.junit.rules.TemporaryFolder;

public class GrapeAndGroovyShellConcurrencyTest {

     @Rule
     public TemporaryFolder tempFolder = new TemporaryFolder();

     private volatile boolean testPassed;

     @Before
     public void setUp() {
         testPassed = true;
     }

     public static void setFileText(File file, String text)
             throws FileNotFoundException, UnsupportedEncodingException {
         PrintWriter writer = new PrintWriter(file, "UTF-8");
         writer.write(text);
         writer.close();
     }

     @Test
     public void testConcurrency() throws Exception {

         File dir = tempFolder.getRoot();
         final int n = 100;
         for (int i=0; i<n; i++) {
             File f1 = new File(dir, "Util" + i + ".groovy");
             setFileText(f1, "@Grab('com.google.guava:guava:18.0')\n"
                     + "import com.google.common.base.Ascii\n"
                     + "class Util" + i + " {\n"
                     + "  static boolean isUpperCase(def c) {\n"
                     + "    return Ascii.isUpperCase(c)\n"
                     + "  }\n"
                     + "}\n"
                     );
         }

         List<Thread> scriptThreads = new LinkedList<Thread>();
         for (int i=0; i<n; i++) {
             final GroovyShell shell = new GroovyShell();
shell.getClassLoader().addClasspath(dir.getAbsolutePath());

             final int j = i;

             Thread scriptThread = new Thread(
                     new Runnable() {
                         public void run() {
                             try {
                                 assertEquals(true, 
shell.evaluate("return Util" + j + ".isUpperCase('C' as char)"));
                                 System.out.println("Thread " + 
Thread.currentThread().getName() + " : OK");
                             } catch (Throwable t) {
                                 System.out.println("Thread " + 
Thread.currentThread().getName() + " : FAILED");
                                 t.printStackTrace();
                                 testPassed = false;
                             }
                         }
                     });
             scriptThread.setDaemon(true);
             scriptThread.setName("run-" + i);
             scriptThread.start();
             scriptThreads.add(scriptThread);
         }

         for (Thread scriptThread : scriptThreads) {
             scriptThread.join();
         }

         assertTrue("must be true", testPassed);
     }

}

------------
Stacktrace 1
------------

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup 
failed:
General error during conversion: java.util.ConcurrentModificationException

java.util.ConcurrentModificationException
     at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859)
     at java.util.ArrayList$Itr.next(ArrayList.java:831)
     at 
org.apache.ivy.util.MessageLoggerHelper.sumupProblems(MessageLoggerHelper.java:45)
     at 
org.apache.ivy.util.MessageLoggerEngine.sumupProblems(MessageLoggerEngine.java:136)
     at org.apache.ivy.util.Message.sumupProblems(Message.java:143)
     at 
org.apache.ivy.core.resolve.ResolveEngine.resolve(ResolveEngine.java:347)
     at org.apache.ivy.Ivy.resolve(Ivy.java:523)
     at org.apache.ivy.Ivy$resolve$1.call(Unknown Source)
     at groovy.grape.GrapeIvy.getDependencies(GrapeIvy.groovy:404)
     at sun.reflect.GeneratedMethodAccessor671.invoke(Unknown Source)
     at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
     at java.lang.reflect.Method.invoke(Method.java:606)
     at 
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSite.invoke(PogoMetaMethodSite.java:166)
     at 
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:56)
     at groovy.grape.GrapeIvy.resolve(GrapeIvy.groovy:563)
     at groovy.grape.GrapeIvy$resolve$56.callCurrent(Unknown Source)
     at groovy.grape.GrapeIvy.resolve(GrapeIvy.groovy:532)
     at groovy.grape.GrapeIvy$resolve$45.callCurrent(Unknown Source)
     at 
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49)
     at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:151)
     at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:179)
     at groovy.grape.GrapeIvy.grab(GrapeIvy.groovy:254)
     at groovy.grape.Grape.grab(Grape.java:163)
     at 
groovy.grape.GrabAnnotationTransformation.visit(GrabAnnotationTransformation.java:358)
     at 
org.codehaus.groovy.transform.ASTTransformationVisitor$3.call(ASTTransformationVisitor.java:319)
     at 
org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:928)
     at 
org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:590)
     at 
org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:566)
     at 
org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:543)
     at 
groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:297)
     at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:267)
     at groovy.lang.GroovyShell.parseClass(GroovyShell.java:692)
     at groovy.lang.GroovyShell.parse(GroovyShell.java:704)
     at groovy.lang.GroovyShell.evaluate(GroovyShell.java:588)
     at groovy.lang.GroovyShell.evaluate(GroovyShell.java:627)
     at groovy.lang.GroovyShell.evaluate(GroovyShell.java:598)
     at 
GrapeAndGroovyShellConcurrencyTest$1.run(GrapeAndGroovyShellConcurrencyTest.java:64)
     at java.lang.Thread.run(Thread.java:745)

1 error

     at 
org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:309)
     at 
org.codehaus.groovy.control.ErrorCollector.addException(ErrorCollector.java:155)
     at 
org.codehaus.groovy.control.SourceUnit.addException(SourceUnit.java:345)
     at 
groovy.grape.GrabAnnotationTransformation.visit(GrabAnnotationTransformation.java:367)
     at 
org.codehaus.groovy.transform.ASTTransformationVisitor$3.call(ASTTransformationVisitor.java:319)
     at 
org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:928)
     at 
org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:590)
     at 
org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:566)
     at 
org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:543)
     at 
groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:297)
     at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:267)
     at groovy.lang.GroovyShell.parseClass(GroovyShell.java:692)
     at groovy.lang.GroovyShell.parse(GroovyShell.java:704)
     at groovy.lang.GroovyShell.evaluate(GroovyShell.java:588)
     at groovy.lang.GroovyShell.evaluate(GroovyShell.java:627)
     at groovy.lang.GroovyShell.evaluate(GroovyShell.java:598)
     at 
GrapeAndGroovyShellConcurrencyTest$1.run(GrapeAndGroovyShellConcurrencyTest.java:64)
     at java.lang.Thread.run(Thread.java:745)

------------
Stacktrace 2
------------

org.codehaus.groovy.control.MultipleCompilationErrorsException: startup 
failed:
file:/var/folders/38/r0n49vmn7zg5dffk79_tgpl80000gn/T/junit124846036508580912/Util7.groovy:

1: unable to resolve class com.google.common.base.Ascii
  @ line 1, column 1.
    @Grab('com.google.guava:guava:18.0')
    ^

1 error

     at 
org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:309)
     at 
org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:943)
     at 
org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:590)
     at 
org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:539)
     at 
groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:297)
     at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:267)
     at groovy.lang.GroovyShell.parseClass(GroovyShell.java:692)
     at groovy.lang.GroovyShell.parse(GroovyShell.java:704)
     at groovy.lang.GroovyShell.evaluate(GroovyShell.java:588)
     at groovy.lang.GroovyShell.evaluate(GroovyShell.java:627)
     at groovy.lang.GroovyShell.evaluate(GroovyShell.java:598)
     at 
GrapeAndGroovyShellConcurrencyTest$1.run(GrapeAndGroovyShellConcurrencyTest.java:64)
     at java.lang.Thread.run(Thread.java:745)

Mime
View raw message