groovy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cédric Champeau <>
Subject Re: Towards a better compiler
Date Sun, 23 Apr 2017 15:08:08 GMT
2017-04-21 22:02 GMT+02:00 Jochen Theodorou <>:

> On 21.04.2017 19:46, Cédric Champeau wrote:
> [...]
>> The fact is that the compiler, for a
>> specific set of sources + compile classpath, should always produce the
>> same output. Especially on the same machine. If it doesn't, then the
>> compiler is not predictable. It's a bug that needs to be fixed. It has
>> nothing to do with Gradle, and *all* users would benefit from this.
> I am curious. How does a user exactly benefit from it outside of a build
> tool like gradle?

Any tool that performs bytecode analysis, or computes signatures is
affected. So if the compiler is not reproducible, since your at the bottom
of the execution, you affect all tools that are built on top. This affects
Gradle, but also Grails, IntelliJ (because different bytecode forces
reindexing), tools which perform bytecode decoration (intercepting
Foo#foo() is not the same as intercepting Bar#foo()) so it also affects

> Also I doubt we can be 100% reproducible as long as we use reflection
> somewhere, simple because reflection is not stable. Only because of the
> great contribution from intellij to be able read class files directly
> without reflection in the compiler we have the ability for *most* classes
> to have a stable result depending on the occurrence in bytecode. But for
> for example the Groovy classes directly known by the compiler like Closure,
> this won´t be the case.
> 100% stability is not achievable simply by using linked maps and sets.

I agree, but I don't get the point of not trying to do better. It should be
pretty obvious that for the same sources and same compile classpath, we
should produce the same output. I don't get why we wouldn't try to fix
this, even incrementally, as we found bugs. This is better. I agree that we
cannot do 100% reproducibility today, because we don't implement
equals/hashcode, which means we're system dependent (and probably JDK
dependent too). And implementing those is doing to be hard, because our
structures are not immutable (so mutating something that is already in the
set/map is problematic). Anyway, I'm not asking for 100% reproducible
today. I'm asking for every dev to realize that we have a problem that
leads to non deterministic behavior, and that everyone should try to take
that into consideration when writing code, and futhermore, tests.

> once you include reflection here... Those cases you could solve by sorting
> though.

Yes, we should avoid reflection as much as possible. I wonder if there are
still lots of places in the compiler where we do this: reflection means
loading the classes and too often initializing, which is probably no good
in any case.

> Especially, order of interfaces in
>> declaration type matter (and they are reproducible today), We *must not*
>> reorder them, or it would change semantics (typically for traits). I
>> have fixed the bugs we, in the Gradle team, have identified, but my
>> email was there to mention that we should take better care of this,
>> because we do a pretty bad job at checking that the behavior of the
>> compiler is deterministic.
> well, that was no real option for a long time and no target till now. But
> if you say your fixes are enough for gradle, then your tests should ensure
> this stays. And then we don´t do a bad job at it anymore, do we?

I have added tests for the cases I fixed, yes. They are not cross-platform,

> Regarding AST transformations classpath, I had actually forgotten that
>> Gradle integrates directly with the compiler, so can use the 2 different
>> "classpath". But AFAIK, our CLI doesn't. This should be easy to fix.
> easy fix? depends... first of all... what will you gain from that? Since I
> have no idea where you get an advantage from those things outside a build
> tool that goes beyond something like make and ant, I fail to see the
> advantage

There would be an advantage for the Groovy compiler itself: today, if you
use `groovyc`, you put everything on the compile classpath. We don't make
the difference between classes that need to be found for implementation of
the compiled classes, and those which are dependencies of AST
transformations. The consequence is the same as the one people have with
annotation processors when on the compile classpath instead of the
annotation processor path: mixing classpath which have nothing to do
together. In particular, say the user code requires Guava 19, but an AST
transformation was built using Guava 17. Then you have a conflict, but the
user shouldn't care about this, because Guava 17 is an implementation
dependency of an AST transformation: nothing that should prevent them from
using the class they wish, because once compiled, Guava 17 is not required
anymore. It's about better modelling of dependencies, which has direct
consequences on user pain.

> Granted, something like groovydoc gets an advantage, but honestly I do not
> see why there it should be done without sorting to achieve a somewhat
> stable output. And of course that sorting can very well happen in the doc
> tool
> bye Jochen

View raw message