systemml-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luciano Resende <luckbr1...@gmail.com>
Subject Re: Java compiler for code generation
Date Mon, 03 Apr 2017 18:27:18 GMT
Is dependency really an issue today, particularly when we bundle the
dependencies with the SystemML jar ? I'd rather include a dependency then
reinventing the wheel and create some code again (unless the dependency
code is flawed).

Also, +1 for continuously reviewing / updating / triming out dependencies.

On Mon, Apr 3, 2017 at 11:04 AM, <dusenberrymw@gmail.com> wrote:

> Using Janino sounds like a great idea.  As for the footprint size for
> Java-only execution modes, it might make sense to do an audit of our
> current dependencies to see if anything can be removed to make up for the
> additional amount.  Then we could just use it in all scenarios without
> worry.
>
> --
>
> Mike Dusenberry
> GitHub: github.com/dusenberrymw
> LinkedIn: linkedin.com/in/mikedusenberry
>
> Sent from my iPhone.
>
>
> > On Mar 31, 2017, at 9:25 PM, Matthias Boehm <mboehm7@googlemail.com>
> wrote:
> >
> > that is a good question. Yes, if we want to enable code generation in
> such
> > a scenario it would also need Janino, which increases our footprint by
> > roughly 0.6MB.
> >
> > Btw, Janino fits much better into such an in-memory deployment because it
> > compiles classes in-memory without the need to write class files into a
> > local working directory. The same could be done for
> > javax.tools.JavaCompiler, but would require to custom in-memory
> > JavaFileManager.
> >
> > Regards,
> > Matthias
> >
> > On Fri, Mar 31, 2017 at 9:14 PM, Berthold Reinwald <reinwald@us.ibm.com>
> > wrote:
> >
> >> Sounds like a good idea.
> >>
> >> Wrt codegen, in a pure Java scoring environment w/o Spark and Hadoop,
> will
> >> the dependency on Janino still be there (that question applies to JDK as
> >> well), and what is the footprint?
> >>
> >> Regards,
> >> Berthold Reinwald
> >> IBM Almaden Research Center
> >> office: (408) 927 2208; T/L: 457 2208
> >> e-mail: reinwald@us.ibm.com
> >>
> >>
> >>
> >> From:   Matthias Boehm <mboehm7@googlemail.com>
> >> To:     dev@systemml.incubator.apache.org
> >> Date:   03/31/2017 08:17 PM
> >> Subject:        Java compiler for code generation
> >>
> >>
> >>
> >> Hi all,
> >>
> >> currently, our new code generator for operator fusion, uses the
> >> programmatic javax.tools.JavaCompiler, which is Java's standard API for
> >> compilation. Despite a plan cache that mitigates unnecessary compilation
> >> and recompilation overheads, we still see significant end-to-end
> overhead
> >> especially for small input data.
> >>
> >> Moving forward, I'd like to switch to Janino
> >> (org.codehaus.janino.SimpleCompiler), which is a fast in-memory Java
> >> compiler with restricted language support. The advantages are
> >>
> >> (1) Reduced compilation overhead: On end-to-end scenarios for L2SVM,
> GLM,
> >> and MLogreg, Janino improved total javac compilation time from 2.039 to
> >> 0.195 (14 operators), from 8.134 to 0.411 (82 operators), and from 4.854
> >> to
> >> 0.283 (46 operators), respectively. At the same time, there was no
> >> measurable impact on runtime efficiency, but even slightly reduced JIT
> >> compilation overhead.
> >>
> >> (2) Removed JDK requirement: Using the standard javax.tools.JavaCompiler
> >> requires the existence of a JDK, while Janino only requires a JRE, which
> >> means it makes it easier to apply code generation by default.
> >>
> >> However, I'm raising this here as Janino would add another explicit
> >> dependency (with BSD license). Fortunately, Spark also uses Janino for
> >> whole-stage-codegen. So we should be able to mark Janino as provided
> >> library. The only issue is a pure Hadoop environment, where we still
> want
> >> to use code generation for CP operations. To simplify the build, I could
> >> imagine using the javax.tools.JavaCompiler for hadoop execution types,
> but
> >> Janino by default.
> >>
> >> If you have any concerns, please let me know by Monday; otherwise I'd
> like
> >> to push this change into our upcoming 0.14 release.
> >>
> >>
> >> Regards,
> >> Matthias
> >>
> >>
> >>
> >>
> >>
>



-- 
Luciano Resende
http://twitter.com/lresende1975
http://lresende.blogspot.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message