groovy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Graeme Rocher <graeme.roc...@gmail.com>
Subject Re: Towards a better compiler
Date Fri, 21 Apr 2017 12:24:52 GMT
Big +1 one for making the compiler reproducible.

I think the usage of Sets and HashMap has an impact on Groovydoc too
because Groovydoc generates different output each time it is run.

For example the "extends" and "implements" output in Groovydoc changes
the order of the classes each time it is run. This must be down to
internal use of unordered sets or hash maps.

Regarding classpath (compiler vs compile classpath), in many cases AST
transforms reference classes from libraries that are on the "compile"
classpath. How would you deal with this case? Have them in both
places?

Cheers

On Sun, Apr 9, 2017 at 11:53 AM, C├ędric Champeau <cchampeau@apache.org> wrote:
> Hi team!
>
> I would like to setup some additional goals for the next release of Groovy.
> As you may know, Gradle 3.5, released tomorrow, will ship with a build cache
> [1]. But Groovy is causing us some troubles, because the output of
> compilation is not reproducible. In other words, for the same inputs, we can
> randomly get a different output. To be clear, while the output of Groovy is
> semantically correct, we cannot guarantee that for the same sources, the
> same bytecode, byte to byte, is going to be generated. This is a big problem
> for Gradle, because it affects the cache key, and a cache miss has terrible
> consequences: since we need to rebuild everything when a build script
> classpath changes, a change in the output of a build script bytecode means
> we need to invalidate a full build...
>
> As an illustration, I fixed 2 issues this week [2] and [3]. But it's not
> enough. We should revise our usage of maps and sets, and use their linked
> counterparts when it makes sense. It, alone, cannot guarantee that we have
> reproducible builds. In particular, cross platform. There are hundreds of
> places where we use hash sets/maps, with objects that do not implement
> equals/hashcode, for example (and since those structures are mutable, we're
> very lucky because the default hashcode uses the system one, which is
> immutable).
>
> Eventually, if you have read my blog post about performance of Gradle 3.4
> [4], you would understand that the Java compiler does a much better job than
> we do as separating things required by the compiler from things required in
> user space. In particular, we at Groovy offer AST transformations, which is
> similar, but not equivalent, to what annotation processors are for javac.
> The problem is that Groovy only has a single compile classpath, which
> includes both the "compiler plugins" (AST transformations) and
> implementation dependencies of the project we compile. So we're effectively
> mixing things that shouldn't be mixed:
>
> - annotations for AST transformations should be on the compile classpath
> - implementation of the AST transformations should be on the AST
> transformations path (compiler classpath)
>
> If we don't do this, we cannot be as smart as what we do in the Java world,
> and compute what is relevant in terms of ABI (application binary interface).
> So any change to the classpath, needs to be considered a breaking change and
> we need to recompile everything. This makes the implementation of a Groovy
> incremental compiler effectively impossible. Furthermore, it prevents the
> AST transform implementors to use the libraries they want as dependencies,
> without leaking them to the user compile classpath (imagine an AST xform
> which uses Guava 18, while the user wants Guava 17 on his classpath). In
> practice, the implementation dependencies of the AST xform are only required
> at compile time, not runtime, so they should be separate.
>
> In short, I vote for the adoption of a new `-compilerpath` where we would
> put everything that affects the compiler itself, and live `--classpath` for
> the user space. Doing this would let tools like Gradle be much smarter about
> what they can do.
>
> [1] https://docs.gradle.org/3.5-rc-3/release-notes.html
> [2] https://issues.apache.org/jira/browse/GROOVY-8142
> [3] https://issues.apache.org/jira/browse/GROOVY-8148
> [4] https://blog.gradle.org/incremental-compiler-avoidance



-- 
Graeme Rocher

Mime
View raw message