From notifications-return-16098-archive-asf-public=cust-asf.ponee.io@groovy.apache.org Thu Apr 11 05:52:03 2019 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [207.244.88.153]) by mx-eu-01.ponee.io (Postfix) with SMTP id 6A84A180784 for ; Thu, 11 Apr 2019 07:52:03 +0200 (CEST) Received: (qmail 81726 invoked by uid 500); 11 Apr 2019 05:52:02 -0000 Mailing-List: contact notifications-help@groovy.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@groovy.apache.org Delivered-To: mailing list notifications@groovy.apache.org Received: (qmail 81660 invoked by uid 99); 11 Apr 2019 05:52:02 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Apr 2019 05:52:02 +0000 Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id AF5F0E2AB1 for ; Thu, 11 Apr 2019 05:52:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id EC5172459C for ; Thu, 11 Apr 2019 05:52:00 +0000 (UTC) Date: Thu, 11 Apr 2019 05:52:00 +0000 (UTC) From: "Daniel Sun (JIRA)" To: notifications@groovy.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Comment Edited] (GROOVY-9068) GroovyScriptEngine causes Metaspace OOM MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/GROOVY-9068?page=3Dcom.atlassia= n.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D168= 15089#comment-16815089 ]=20 Daniel Sun edited comment on GROOVY-9068 at 4/11/19 5:51 AM: ------------------------------------------------------------- I tried to execute {{(1..10).sum()}} for 10000 times with the default JVM h= eap size, everything is OK. {code:java} def sh =3D new GroovyShell() 10000.times { =09sh.evaluate "(1..10).sum()" } {code} The following code you provided is not thread safe. {code:java} Class getScriptClass(String script) throws CompilationFailedException { Class clazz =3D classMap.get(script); if (clazz !=3D null) { return clazz; } // The breakpoint is below line clazz =3D loader.parseClass(script, generateScriptName()); classMap.put(script, clazz); return clazz; } {code} was (Author: daniel_sun): I tried to execute {{(1..10).sum()}} for 10000 times with the default JVM h= eap size, everything is OK. {code:java} def sh =3D new GroovyShell() 10000.times { =09sh.evaluate "(1..10).sum()" } {code} The following code you provided is not thread safe. {code:java} Class getScriptClass(String script) throws CompilationFailedException { Class clazz =3D classMap.get(script); if (clazz !=3D null) { return clazz; } // The breakpoint is below line clazz =3D loader.parseClass(script, generateScriptName()); classMap.put(script, clazz); return clazz; } {code} Please try the following code with Groovy 2.5.6: {code:java} private static final GroovyClassLoader GCL =3D new GroovyClassLoader(); public static Object evaluate(String scriptText) { Class scriptClass =3D GCL.parseClass(scriptText); return InvokerHelper.createScript(scriptClass, new Binding()).run()= ; } {code} > GroovyScriptEngine causes Metaspace OOM > --------------------------------------- > > Key: GROOVY-9068 > URL: https://issues.apache.org/jira/browse/GROOVY-9068 > Project: Groovy > Issue Type: Bug > Components: GroovyScriptEngine > Affects Versions: 2.4.9 > Environment: macOS Mojave, MacBook Pro (Retina, 15-inch, Mid 2015= ) > Reporter: Jingfei Hu > Priority: Major > Labels: GroovyScriptEngineImpl, Metaspace, OOM > > Hello team, > We've encountered this troublesome Metaspace OOM in our application recen= tly as the number of groovy scripts increases. The groovy usage pattern in = our application is evaluating scripts adhoc and directly. There are no any = kinds of caches. And we use below code to do the evaluation.=C2=A0 > =C2=A0 > {code:java} > engine.eval(scriptText, bindings); > {code} > We thought the cache of GroovyScriptEngineImpl which is below field would= take effect, but actually not in the multi-threading=C2=A0context > {code:java} > // script-string-to-generated Class map > private ManagedConcurrentValueMap classMap =3D new Managed= ConcurrentValueMap(ReferenceBundle.getSoftBundle()); > {code} > So without proper cache, our application continuously leverages the groov= y class loader of=C2=A0GroovyScriptEngineImpl to parse scripts to generate = Class objects and fill up Metaspace eventually.=C2=A0 > =C2=A0 > And below code snippets can easily reproduce this.=C2=A0=C2=A0 > =C2=A0 > {code:java} > package com.jingfei; > import java.util.concurrent.ExecutorService; > import java.util.concurrent.Executors; > public class Main { > static final GroovyScriptExecutor groovyScriptExecutor =3D new Groovy= ScriptExecutor(); > public static void main(String[] args) throws InterruptedException { > ExecutorService executor =3D Executors.newFixedThreadPool(10000); > while (true) { > executor.execute(new Runnable() { > @Override > public void run() { > try { > groovyScriptExecutor.executeScript("(1..10).sum()= "); > } catch (Exception e) { > e.printStackTrace(); > } > } > }); > } > } > } > package com.jingfei; > import java.util.HashMap; > import java.util.Map; > import javax.script.CompiledScript; > import javax.script.ScriptEngine; > import javax.script.ScriptEngineManager; > import javax.script.ScriptException; > import groovy.lang.GroovyClassLoader; > import org.codehaus.groovy.jsr223.GroovyCompiledScript; > import org.codehaus.groovy.jsr223.GroovyScriptEngineImpl; > /** > * @author jingfei > * @version $Id: GroovyScriptExecutor.java, v 0.1 2019-04-02 20:07 jingfe= i Exp $$ > */ > public class GroovyScriptExecutor { > /** Script Engine Manager */ > private static final ScriptEngineManager factory =3D new ScriptEngine= Manager(); > /** Script engine */ > private static final ScriptEngine engine =3D factory.getEngineByName= ("groovy"); > public void executeScript(final String scriptText) throws ScriptExcep= tion { > System.out.println(engine.eval(scriptText)); > } > }{code} > Looking into the Metaspace dump, we find out within the Metaspace there a= re hundreds of class loader objects named=C2=A0*groovy/lang/GroovyClassLoad= er$InnerLoader*=C2=A0and all of Class meta info loaded by them with the nam= ing pattern=C2=A0*ScriptXXXX.* > =C2=A0 > Since the script is all the same, i.e.=C2=A0 > {code:java} > (1..10).sum(){code} > , this behavior=C2=A0seems totally odd to me, because I thought due to th= e existence of the cache, there would be only one class loader created nece= ssary to parse the script.=C2=A0 > =C2=A0 > Any help is appreciated to work out this problem! Thanks! -- This message was sent by Atlassian JIRA (v7.6.3#76005)