groovy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cédric Champeau <cchamp...@apache.org>
Subject Re: Towards a better compiler
Date Fri, 21 Apr 2017 17:49:37 GMT
2017-04-21 14:24 GMT+02:00 Graeme Rocher <graeme.rocher@gmail.com>:

> Big +1 one for making the compiler reproducible.
>
> I think the usage of Sets and HashMap has an impact on Groovydoc too
> because Groovydoc generates different output each time it is run.
>
> For example the "extends" and "implements" output in Groovydoc changes
> the order of the classes each time it is run. This must be down to
> internal use of unordered sets or hash maps.
>
> Regarding classpath (compiler vs compile classpath), in many cases AST
> transforms reference classes from libraries that are on the "compile"
> classpath. How would you deal with this case? Have them in both
> places?
>

Yes, they should. Knowing that if you do so, it implies that a potential
incremental compiler would be defeated.

>
> Cheers
>
> On Sun, Apr 9, 2017 at 11:53 AM, Cédric Champeau <cchampeau@apache.org>
> wrote:
> > Hi team!
> >
> > I would like to setup some additional goals for the next release of
> Groovy.
> > As you may know, Gradle 3.5, released tomorrow, will ship with a build
> cache
> > [1]. But Groovy is causing us some troubles, because the output of
> > compilation is not reproducible. In other words, for the same inputs, we
> can
> > randomly get a different output. To be clear, while the output of Groovy
> is
> > semantically correct, we cannot guarantee that for the same sources, the
> > same bytecode, byte to byte, is going to be generated. This is a big
> problem
> > for Gradle, because it affects the cache key, and a cache miss has
> terrible
> > consequences: since we need to rebuild everything when a build script
> > classpath changes, a change in the output of a build script bytecode
> means
> > we need to invalidate a full build...
> >
> > As an illustration, I fixed 2 issues this week [2] and [3]. But it's not
> > enough. We should revise our usage of maps and sets, and use their linked
> > counterparts when it makes sense. It, alone, cannot guarantee that we
> have
> > reproducible builds. In particular, cross platform. There are hundreds of
> > places where we use hash sets/maps, with objects that do not implement
> > equals/hashcode, for example (and since those structures are mutable,
> we're
> > very lucky because the default hashcode uses the system one, which is
> > immutable).
> >
> > Eventually, if you have read my blog post about performance of Gradle 3.4
> > [4], you would understand that the Java compiler does a much better job
> than
> > we do as separating things required by the compiler from things required
> in
> > user space. In particular, we at Groovy offer AST transformations, which
> is
> > similar, but not equivalent, to what annotation processors are for javac.
> > The problem is that Groovy only has a single compile classpath, which
> > includes both the "compiler plugins" (AST transformations) and
> > implementation dependencies of the project we compile. So we're
> effectively
> > mixing things that shouldn't be mixed:
> >
> > - annotations for AST transformations should be on the compile classpath
> > - implementation of the AST transformations should be on the AST
> > transformations path (compiler classpath)
> >
> > If we don't do this, we cannot be as smart as what we do in the Java
> world,
> > and compute what is relevant in terms of ABI (application binary
> interface).
> > So any change to the classpath, needs to be considered a breaking change
> and
> > we need to recompile everything. This makes the implementation of a
> Groovy
> > incremental compiler effectively impossible. Furthermore, it prevents the
> > AST transform implementors to use the libraries they want as
> dependencies,
> > without leaking them to the user compile classpath (imagine an AST xform
> > which uses Guava 18, while the user wants Guava 17 on his classpath). In
> > practice, the implementation dependencies of the AST xform are only
> required
> > at compile time, not runtime, so they should be separate.
> >
> > In short, I vote for the adoption of a new `-compilerpath` where we would
> > put everything that affects the compiler itself, and live `--classpath`
> for
> > the user space. Doing this would let tools like Gradle be much smarter
> about
> > what they can do.
> >
> > [1] https://docs.gradle.org/3.5-rc-3/release-notes.html
> > [2] https://issues.apache.org/jira/browse/GROOVY-8142
> > [3] https://issues.apache.org/jira/browse/GROOVY-8148
> > [4] https://blog.gradle.org/incremental-compiler-avoidance
>
>
>
> --
> Graeme Rocher
>

Mime
View raw message