groovy-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cédric Champeau <cedric.champ...@gmail.com>
Subject A reminder about how things are compiled
Date Wed, 27 Dec 2017 09:04:44 GMT
Hi fellow Groovy contributors!

Given the recent question from Jochen about how to only execute tests from
the "core" project when you "know you only modified core" triggering
re-execution of tests for all modules, let me explain why this is like this.

Groovy is partially written in Groovy. It means that we have source code
written in Java, and source code written in Groovy. The code written in
Java is mostly, but not limited to, the "minimal compiler infrastructure".
This means that if you only compile the Java sources of the Groovy
codebase, what you obtain is something that is capable of compiling Groovy
code (and a bit more).

For this reason, we have decided, a few years back, to compile Groovy
sources using this "minimal compiler", that we call the "bootstrap
compiler". Said differently, not only Groovy is partially written in
Groovy, but we also compile the Groovy sources using the compiler generated
by the _current sources_.

Some other projects choose a different strategy: they compile, say, Java,
with version N-1 of Java. Groovy compiles itself with the _same_ version.
This has several advantages, like the fact that if Groovy N-1 produces
wrong bytecode, for some reason, we can immediately fix it. Similarly, the
generated bytecode is consistent with the bytecode that users will have
when using Groovy. Eventually, it also "accidentally" increases our test
coverage as a bug in the compiler is very likely to fail our build.

The consequence, however, is that any change to a Java class in Groovy core
is going to produce a different compiler. For Gradle, which is aware of
inputs/outputs, it means that the compiler has changed, and that it needs
to recompile downstream consumers (subprojects) and, of course, re-execute
their tests. This is the _correct behavior_. Gradle is right to do so,
because the compiler has changed, so the bytecode generated might be
different, but also since Groovy provides a runtime, it also needs to
re-execute the tests because the runtime has changed.

What I explain is also true of the other tasks we use, like groovydoc, or
docgenerator. Now, let me explain why changing the strategy to use compiler
N-1 is not necessarily a good idea for us: as I explained, Groovy also
comes with a runtime. Say that in Groovy 3, we decide to get rid of call
site caching, to only use invokedynamic. Then it means that the runtime of
Groovy 3 will no longer include call site caching. However, the Groovy
classes of the compiler would have been compiled with call site caching, so
a _consumer_ of the compiler would fail, because those classes would no
longer be there at runtime!

Of course one might say "then you can use the invokedynamic version" of
Groovy to compile Groovy 3, which leads to the last bit of complexity of
our build. Some would have noticed that we now have a "testAll" task for
each project. This task executes tests with the "indy" version of the
compiler. Which means that in practice, we produce 2 versions of the
compiler, not just one. This was the main reason for the complexity of the
previous build, that I recently got rid of by using a different strategy
and leveraging the Gradle build cache. So, instead of using the same
outputs for both compilers, they are now separate, and we can run the tests
in 2 flavors. The consequence is that tests are executed twice (one for
`test`, the other for `testWithIndy`), but the outcome is much cleaner.

I hope this clarifies things a bit. Now for daily development, you can use:

./gradlew :test : will only execute the call site caching version of tests
for the "core" project
./gradlew :testWithIndy : will only execute the indy version of tests for
the "core" project
./gradlew :testAll : will execute both flavors of tests (indy and non indy)
for the "core" project

And of course you can do the same for any subproject:

./gradlew :groovy-xml:test

You can also choose precisely which test to execute by adding `--tests
*MyTest*` to the command line.

Cheers,
Cédric

Mime
View raw message