Mailing-List: contact dev-help@groovy.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@groovy.apache.org
Subject: Re: A reminder about how things are compiled
To: dev@groovy.apache.org
References: <CADQzvmkpwb48P0tvFyo0Tu-0xu1rbEH8pLSsQ=1zXzsnVBMfbQ@mail.gmail.com>
From: Jochen Theodorou <blackdrag@gmx.org>
Message-ID: <38b4c011-e43b-92e1-1489-3c663e7c787d@gmx.org>
Date: Wed, 27 Dec 2017 13:22:42 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.1.1
MIME-Version: 1.0
In-Reply-To: <CADQzvmkpwb48P0tvFyo0Tu-0xu1rbEH8pLSsQ=1zXzsnVBMfbQ@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
archived-at: Wed, 27 Dec 2017 12:22:50 -0000

On 27.12.2017 10:04, Cédric Champeau wrote:
[...]
> The consequence, however, is that any change to a Java class in Groovy 
> core is going to produce a different compiler.

This is fine in the IDE, as long as I do not have to bootstrap 
immediately... which I do not have to.

But I must say the IDEA setup is still annoying as hell. I freshly 
generated the modules and ended up with a project I cannot compile, 
because of the examples. I then switched to build, but ignore errors, 
which resulted in my main (FileSystemCompiler, GroovyMain) classes not 
being found, because they have not been compiled and I did not add the 
Groovy global lib. Then I added the bootstrap and while the main classes 
now have been found, they have been of course from the bootstrap, not 
the source. I will always have to have a really, really, really good 
look at things to see if I am using bootstrap or not. I mean imagine you 
remove a class and then run from the IDE to see if your test still 
works. And it will, because the damn class is still in the bootstrap 
jar, which of course you did not create a new one. I easily lost more 
than half a day just trying to setup things again. Together with the 
installation and fixing of intellij itself... that was no fun day at all 
and lost time for Groovy development.

> For Gradle, which is 
> aware of inputs/outputs, it means that the compiler has changed, and 
> that it needs to recompile downstream consumers (subprojects) and, of 
> course, re-execute their tests. This is the _correct behavior_. Gradle 
> is right to do so, because the compiler has changed, so the bytecode 
> generated might be different, but also since Groovy provides a runtime, 
> it also needs to re-execute the tests because the runtime has changed.
> 
> What I explain is also true of the other tasks we use, like groovydoc, 
> or docgenerator.

that part is perfectly fine.

> Now, let me explain why changing the strategy to use 
> compiler N-1 is not necessarily a good idea for us: as I explained, 
> Groovy also comes with a runtime. Say that in Groovy 3, we decide to get 
> rid of call site caching, to only use invokedynamic. Then it means that 
> the runtime of Groovy 3 will no longer include call site caching. 
> However, the Groovy classes of the compiler would have been compiled 
> with call site caching, so a _consumer_ of the compiler would fail, 
> because those classes would no longer be there at runtime!
> 
> Of course one might say "then you can use the invokedynamic version" of 
> Groovy to compile Groovy 3, which leads to the last bit of complexity of 
> our build.

What you normally do is compiler with N-1 to get a compiler for N' and 
then use that compiler to get the real N. Very common strategy. Of 
course that means in a build where we would depend on an older release 
(N-1), we not be able to use features in the new version (N') before we 
have created N', which can actually compile the new features. And only 
then N could be compiled using the new features from N'. Using Java is 
kind of like saying we stay with N'. So there are pros and cons to this 
approach, it surely does not make things more easy from the build side. 
It would make the program code more easy and would be better in terms of 
"eat you own dog food".

what me really prevents from doing something like this is that the 
static compiler has to many fallbacks to dynamic code where I do not 
want them.

> Some would have noticed that we now have a "testAll" task for 
> each project. This task executes tests with the "indy" version of the 
> compiler. Which means that in practice, we produce 2 versions of the 
> compiler, not just one. This was the main reason for the complexity of 
> the previous build, that I recently got rid of by using a different 
> strategy and leveraging the Gradle build cache. So, instead of using the 
> same outputs for both compilers, they are now separate, and we can run 
> the tests in 2 flavors. The consequence is that tests are executed twice 
> (one for `test`, the other for `testWithIndy`), but the outcome is much 
> cleaner.
> 
> I hope this clarifies things a bit. Now for daily development, you can use:
> 
> ./gradlew :test : will only execute the call site caching version of 
> tests for the "core" project
> ./gradlew :testWithIndy : will only execute the indy version of tests 
> for the "core" project
> ./gradlew :testAll : will execute both flavors of tests (indy and non 
> indy) for the "core" project
> 
> And of course you can do the same for any subproject:
> 
> ./gradlew :groovy-xml:test
> 
> You can also choose precisely which test to execute by adding `--tests 
> *MyTest*` to the command line.

testAll and testWithIndy I did not realize,thanks.

bye Jcohen