Mailing-List: contact dev-help@groovy.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@groovy.incubator.apache.org
Date: Thu, 26 Nov 2015 13:58:34 +0100 (CET)
From: Remi Forax <forax@univ-mlv.fr>
To: dev@groovy.incubator.apache.org
Message-ID: <2131754572.49711.1448542714105.JavaMail.zimbra@u-pem.fr>
In-Reply-To: 
 <CADQzvm=xFamhebBjanaaYOeQRoXOqJ=dJ9AjOa457BtL1Spa9w@mail.gmail.com>
References: <56164EAB.1030302@gmx.org>
 <CAByu6UVLBaJ9Ad8eMqPt2knbn4UTpeDkWGm0bxY+yVwx3SW-HQ@mail.gmail.com>
 <CADQzvmkPunNym4Xra7xn8nvrwghhUgP7Ah06=CwW+e+AZ+MS7w@mail.gmail.com>
 <5648ADB3.9090906@gmx.org>
 <CADQzvm=LX9LXQEoObhEfV-NkaP-c-Dnkw-0B7d2W3-4zKtgbnw@mail.gmail.com>
 <116343119.48441.1448541597230.JavaMail.zimbra@u-pem.fr>
 <CADQzvm=xFamhebBjanaaYOeQRoXOqJ=dJ9AjOa457BtL1Spa9w@mail.gmail.com>
Subject: Re: challenges through Java modules (aka jigsaw)
MIME-Version: 1.0
Content-Type: multipart/alternative;
	boundary="----=_Part_49710_1234926740.1448542714104"
Thread-Topic: challenges through Java modules (aka jigsaw)
Thread-Index: 87tbkPBEsvWrgMdoJ5WT8A8b35Zsog==

------=_Part_49710_1234926740.1448542714104
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

----- Mail original -----

> De: "C=C3=A9dric Champeau" <cedric.champeau@gmail.com>
> =C3=80: dev@groovy.incubator.apache.org
> Envoy=C3=A9: Jeudi 26 Novembre 2015 13:49:18
> Objet: Re: challenges through Java modules (aka jigsaw)

> > You can have breaking changes in the way the classes are seen from Java
> > (and
> > jigsaw) without having breaking changes when you use these classes from
> > Groovy.
>=20
> > By example, you can introduce a way to create type aliases to Groovy, g=
iven
> > that, you can move Java classes around and keep the same references in =
the
> > Groovy code.
>=20

> This would work only if a library is recompiled against the latest Groovy
> version (and even more true if you use @CompileStatic), whereas the idea =
of
> backwards compatibility is *not* to recompile everything. That is to say,
> take Groovy N+1 and it is capable of running classes compiled with Groovy=
 N.

The translation of types can appear at runtime if the type aliases are know=
n by the groovy class loader.=20
I think you only need re-compilation if you use @CompileStatic.=20

> > Also about groovy-all, you can create a module groovy-all that will req=
uire
> > all other modules, no ?
>=20

> Yes, but that's not the purpose of this module today. It's more for guys =
who
> are too lazy to precisely determine all their Groovy module dependencies =
and
> prefer to use groovy-all instead of groovy + "find all modules that I use=
".
> Note that the simplification that I give is recent, because a few release=
s
> ago, groovy-all contained repackaged ASM classes, whereas groovy.jar did
> not. Now both have repackaged classes, making it much more consistent and
> even less useful to use groovy-all.

if it's only packaging, you just have to create a module-info.class that wi=
ll concatenate all the module-info.class of all other modules, you may need=
 ASM for that :)=20

R=C3=A9mi=20

> > regards,
>=20
> > R=C3=A9mi
>=20

> > > De: "C=C3=A9dric Champeau" < cedric.champeau@gmail.com >
> >=20
>=20
> > > =C3=80: dev@groovy.incubator.apache.org
> >=20
>=20
> > > Envoy=C3=A9: Jeudi 26 Novembre 2015 13:10:11
> >=20
>=20
> > > Objet: Re: challenges through Java modules (aka jigsaw)
> >=20
>=20

> > > So I'm taking advantage of a bit of free time to write a quick summar=
y of
> > > the
> > > situation and what we will have to decide. Basically, we have a serio=
us
> > > problem (but we will not be the only ones).
> >=20
>=20

> > > When you declare a Jigsaw module, one has to declare the list of pack=
ages
> > > that it exports (and sub-packages are not exported). Those packages a=
re
> > > the
> > > ones that can be consumed from other Jigsaw modules. It therefore
> > > implements
> > > strong encapsulation, but making it impossible to reference an intern=
al
> > > class of a module, both at compile time and runtime. So far so good. =
The
> > > problem is that the list of packages that belong to a module is exclu=
sive
> > > to
> > > that module. So if module A exports "foo.bar", another module cannot =
do
> > > the
> > > same, or it will fail whenever the 2 modules are loaded at the same t=
ime.
> > > Worse, *internal packages* are also exclusive, so even if you don't
> > > export
> > > a
> > > package, two modules containing the same internal packages will *also=
*
> > > fail.
> >=20
>=20

> > > Why is it a problem for Groovy? It is a problem because we have a ver=
y
> > > long
> > > history. And part of this history makes that modularization was
> > > introduced
> > > pretty recently, in the 2.0 release. Before, Groovy only had a single
> > > module. Now Groovy provides multiple modules, but also a "groovy-all"=
 jar
> > > that contains all the classes from all modules. At this point, we can
> > > already see that this won't be possible anymore: groovy-all must die,=
 or
> > > we
> > > must not support modules (sigh). But the problem is even worse. Since=
 the
> > > split into modules was done late and that we wanted to preserve backw=
ards
> > > compatibility, we decided not to change the packages of classes extra=
cted
> > > into their own modules. So for example, XmlParser is found in groovy.=
util
> > > (not groovy.util.xml) in the groovy-xml module, but we also have lots=
 of
> > > classes that live into groovy.xml into the "groovy" core module. So i=
n
> > > short, our modules are no fit for Jigsaw: we must come back to the ve=
ry
> > > bad
> > > state of having everything in a single module.
> >=20
>=20

> > > Of course, that's not a satisfactory solution, and of course, there's
> > > nothing
> > > we can do to influence the decision not to make such a strong
> > > requirement:
> > > there are technical reasons behind this choice.
> >=20
>=20

> > > So what can we do?
> >=20
>=20

> > > 1. the easiest, fastest path, is to kill all modules that we have tod=
ay,
> > > and
> > > go with a single, good old, groovy-all jar. We would go years backwar=
ds,
> > > and
> > > it's definitely not something we want to do. We want to have *more*
> > > modularization, in particular for Android, where the current split is
> > > still
> > > too big.
> >=20
>=20
> > > 2. refactor modules so that each module has its own set of packages, =
and
> > > hope
> > > that we don't end up with a big groovy-all jar. Seems very unlikely.
> >=20
>=20
> > > 3. break binary compatibility, move classes around, reorganize stuff.
> >=20
>=20

> > > Despite my worst nightmare of fragmenting the community (that is what
> > > happened for all languages that tried to introduce breaking changes),=
 I
> > > think we don't have the choice and we have to go route 3. And if we d=
o
> > > so,
> > > I
> > > would take advantage of this fact to:
> >=20
>=20
> > > 1. reorganize our code base even further to separate everything into
> > > their
> > > own modules and make them as minimal as possible
> >=20
>=20
> > > 2. migrate to Java 8 as the minimum JDK version, which would allow us=
 to
> > > drop
> > > support of the old call site caching, get a new MOP, and possibly
> > > removing
> > > everything that annoys us in terms of behavior which is kept for the =
sake
> > > of
> > > compatibility
> >=20
>=20

> > > It's a long road, and it is pretty obvious that we won't be able to b=
e
> > > ready
> > > for Jigsaw launch (october 2016), but it we don't decide today what w=
e
> > > want
> > > to do, we will soon be obsolete, because nobody will be able to use
> > > Groovy
> > > with Jigsaw.
> >=20
>=20

> > > Last but not least, as part of the Gradle team, I'd like to take
> > > advantage
> > > of
> > > this to rework our build to make it more consistent, faster, and aime=
d
> > > towards the future. In next January, we will start promoting a smooth
> > > migration path to Jigsaw using the Java software model. I think Groov=
y is
> > > still too big to migrate now to the java software model, but at least=
, we
> > > can start investigating. It is crucial to us because Gradle uses Groo=
vy
> > > internally, so if Gradle doesn't run on Jigsaw, we will have a medium
> > > term
> > > problem (medium, because we could still run on JDK 8 but target JDK 9=
).
> >=20
>=20

> > > Let the fun begin!
> >=20
>=20

> > > 2015-11-15 17:07 GMT+01:00 Jochen Theodorou < blackdrag@gmx.org > :
> >=20
>=20

> > > > On 15.11.2015 12:11, C=C3=A9dric Champeau wrote:
> > >=20
> >=20
>=20

> > > > > Hi guys,
> > > >=20
> > >=20
> >=20
>=20

> > > > > After spending some days at Devoxx, I had a few chats about Jigsa=
w
> > > > > and
> > > >=20
> > >=20
> >=20
>=20
> > > > > the implications on Groovy. I will send an email summarizing them=
 and
> > > > > I
> > > >=20
> > >=20
> >=20
>=20
> > > > > think we need to come up with a plan.
> > > >=20
> > >=20
> >=20
>=20

> > > > I was saying that, yes. Seems nobody wanted to believe me
> > >=20
> >=20
>=20

> > > > > I'm not sure JIRA is the best way
> > > >=20
> > >=20
> >=20
>=20
> > > > > to track this, but in short, we have serious issues coming, and
> > > > > breaking
> > > >=20
> > >=20
> >=20
>=20
> > > > > changes. Maybe it's a chance for us to make all the breaking chan=
ges
> > > > > we
> > > >=20
> > >=20
> >=20
>=20
> > > > > thought for so long.
> > > >=20
> > >=20
> >=20
>=20

> > > > I doubt it, we don't have the manpower, but we can do a bit of cour=
se
> > >=20
> >=20
>=20

> > > > bye Jochen
> > >=20
> >=20
>=20

------=_Part_49710_1234926740.1448542714104
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: quoted-printable

<html><body><div style=3D"font-family: times new roman, new york, times, se=
rif; font-size: 12pt; color: #000000"><div><br></div><div><br></div><hr id=
=3D"zwchr"><blockquote style=3D"border-left:2px solid #1010FF;margin-left:5=
px;padding-left:5px;color:#000;font-weight:normal;font-style:normal;text-de=
coration:none;font-family:Helvetica,Arial,sans-serif;font-size:12pt;"><b>De=
: </b>"C=C3=A9dric Champeau" &lt;cedric.champeau@gmail.com&gt;<br><b>=C3=80=
: </b>dev@groovy.incubator.apache.org<br><b>Envoy=C3=A9: </b>Jeudi 26 Novem=
bre 2015 13:49:18<br><b>Objet: </b>Re: challenges through Java modules (aka=
 jigsaw)<br><div><br></div><div dir=3D"ltr"><div class=3D"gmail_extra"><div=
 class=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0 =
0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style=3D"font-fa=
mily:times new roman,new york,times,serif;font-size:12pt;color:#000000"><br=
><div><br></div><div>You can have breaking changes in the way the classes a=
re seen from Java (and jigsaw) without having breaking changes when you use=
 these classes from Groovy. <br></div><div>By example, you can introduce a =
way to create type aliases to Groovy, given that, you can move Java classes=
 around and keep the same references in the Groovy code.<br></div><div><br>=
</div></div></blockquote><div>This would work only if a library is recompil=
ed against the latest Groovy version (and even more true if you use @Compil=
eStatic), whereas the idea of backwards compatibility is *not* to recompile=
 everything. That is to say, take Groovy N+1 and it is capable of running c=
lasses compiled with Groovy N.</div></div></div></div></blockquote><div><br=
></div><div>The translation of types can appear at runtime if the type alia=
ses are known by the groovy class loader.<br></div><div>I think you only ne=
ed re-compilation if you use @CompileStatic.<br></div><div><br></div><block=
quote style=3D"border-left:2px solid #1010FF;margin-left:5px;padding-left:5=
px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;fon=
t-family:Helvetica,Arial,sans-serif;font-size:12pt;"><div dir=3D"ltr"><div =
class=3D"gmail_extra"><div class=3D"gmail_quote"><blockquote class=3D"gmail=
_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:=
1ex"><div style=3D"font-family:times new roman,new york,times,serif;font-si=
ze:12pt;color:#000000"><div>Also about groovy-all, you can create a module =
groovy-all that will require all other modules, no ?<br></div></div></block=
quote><div><br></div><div>Yes, but that's not the purpose of this module to=
day. It's more for guys who are too lazy to precisely determine all their G=
roovy module dependencies and prefer to use groovy-all instead of groovy + =
"find all modules that I use". Note that the simplification that I give is =
recent, because a few releases ago, groovy-all contained repackaged ASM cla=
sses, whereas groovy.jar did not. Now both have repackaged classes, making =
it much more consistent and even less useful to use groovy-all.</div></div>=
</div></div></blockquote><div><br></div><div>if it's only packaging, you ju=
st have to create a module-info.class that will concatenate all the module-=
info.class of all other modules, you may need ASM for that :)<br></div><div=
><br></div><div>R=C3=A9mi<br></div><blockquote style=3D"border-left:2px sol=
id #1010FF;margin-left:5px;padding-left:5px;color:#000;font-weight:normal;f=
ont-style:normal;text-decoration:none;font-family:Helvetica,Arial,sans-seri=
f;font-size:12pt;"><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=
=3D"gmail_quote"><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8=
ex;border-left:1px #ccc solid;padding-left:1ex"><div style=3D"font-family:t=
imes new roman,new york,times,serif;font-size:12pt;color:#000000"><div><br>=
</div><div>regards,<br></div><div>R=C3=A9mi<br></div><div><br></div><hr><bl=
ockquote style=3D"border-left:2px solid #1010ff;margin-left:5px;padding-lef=
t:5px;color:#000;font-weight:normal;font-style:normal;text-decoration:none;=
font-family:Helvetica,Arial,sans-serif;font-size:12pt"><b>De: </b>"C=C3=A9d=
ric Champeau" &lt;<a href=3D"mailto:cedric.champeau@gmail.com" target=3D"_b=
lank">cedric.champeau@gmail.com</a>&gt;<br><b>=C3=80: </b><a href=3D"mailto=
:dev@groovy.incubator.apache.org" target=3D"_blank">dev@groovy.incubator.ap=
ache.org</a><br><b>Envoy=C3=A9: </b>Jeudi 26 Novembre 2015 13:10:11<br><b>O=
bjet: </b>Re: challenges through Java modules (aka jigsaw)<div><div class=
=3D"h5"><br><div><br></div><div dir=3D"ltr">So I'm taking advantage of a bi=
t of free time to write a quick summary of the situation and what we will h=
ave to decide. Basically, we have a serious problem (but we will not be the=
 only ones).<div><br></div><div>When you declare a Jigsaw module, one has t=
o declare the list of packages that it exports (and sub-packages are not ex=
ported). Those packages are the ones that can be consumed from other Jigsaw=
 modules. It therefore implements strong encapsulation, but making it impos=
sible to reference an internal class of a module, both at compile time and =
runtime. So far so good. The problem is that the list of packages that belo=
ng to a module is exclusive to that module. So if module A exports "foo.bar=
", another module cannot do the same, or it will fail whenever the 2 module=
s are loaded at the same time. Worse, *internal packages* are also exclusiv=
e, so even if you don't export a package, two modules containing the same i=
nternal packages will *also* fail.</div><div><br></div><div>Why is it a pro=
blem for Groovy? It is a problem because we have a very long history. And p=
art of this history makes that modularization was introduced pretty recentl=
y, in the 2.0 release. Before, Groovy only had a single module. Now Groovy =
provides multiple modules, but also a "groovy-all" jar that contains all th=
e classes from all modules. At this point, we can already see that this won=
't be possible anymore: groovy-all must die, or we must not support modules=
 (sigh). But the problem is even worse. Since the split into modules was do=
ne late and that we wanted to preserve backwards compatibility, we decided =
not to change the packages of classes extracted into their own modules. So =
for example, XmlParser is found in groovy.util (not groovy.util.xml) in the=
 groovy-xml module, but we also have lots of classes that live into groovy.=
xml into the "groovy" core module. So in short, our modules are no fit for =
Jigsaw: we must come back to the very bad state of having everything in a s=
ingle module.</div><div><br></div><div>Of course, that's not a satisfactory=
 solution, and of course, there's nothing we can do to influence the decisi=
on not to make such a strong requirement: there are technical reasons behin=
d this choice.</div><div><br></div><div>So what can we do?</div><div><br></=
div><div>1. the easiest, fastest path, is to kill all modules that we have =
today, and go with a single, good old, groovy-all jar. We would go years ba=
ckwards, and it's definitely not something we want to do. We want to have *=
more* modularization, in particular for Android, where the current split is=
 still too big.</div><div>2. refactor modules so that each module has its o=
wn set of packages, and hope that we don't end up with a big groovy-all jar=
. Seems very unlikely.</div><div>3. break binary compatibility, move classe=
s around, reorganize stuff.</div><div><br></div><div>Despite my worst night=
mare of fragmenting the community (that is what happened for all languages =
that tried to introduce breaking changes), I think we don't have the choice=
 and we have to go route 3. And if we do so, I would take advantage of this=
 fact to:</div><div>1. reorganize our code base even further to separate ev=
erything into their own modules and make them as minimal as possible</div><=
div>2. migrate to Java 8 as the minimum JDK version, which would allow us t=
o drop support of the old call site caching, get a new MOP, and possibly re=
moving everything that annoys us in terms of behavior which is kept for the=
 sake of compatibility</div><div><br></div><div>It's a long road, and it is=
 pretty obvious that we won't be able to be ready for Jigsaw launch (octobe=
r 2016), but it we don't decide today what we want to do, we will soon be o=
bsolete, because nobody will be able to use Groovy with Jigsaw.</div><div><=
br></div><div>Last but not least, as part of the Gradle team, I'd like to t=
ake advantage of this to rework our build to make it more consistent, faste=
r, and aimed towards the future. In next January, we will start promoting a=
 smooth migration path to Jigsaw using the Java software model. I think Gro=
ovy is still too big to migrate now to the java software model, but at leas=
t, we can start investigating. It is crucial to us because Gradle uses Groo=
vy internally, so if Gradle doesn't run on Jigsaw, we will have a medium te=
rm problem (medium, because we could still run on JDK 8 but target JDK 9).<=
/div><div><br></div><div>Let the fun begin!</div></div><div class=3D"gmail_=
extra"><br><div class=3D"gmail_quote">2015-11-15 17:07 GMT+01:00 Jochen The=
odorou <span dir=3D"ltr">&lt;<a href=3D"mailto:blackdrag@gmx.org" target=3D=
"_blank">blackdrag@gmx.org</a>&gt;</span>:<br><blockquote class=3D"gmail_qu=
ote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex=
"><span>On 15.11.2015 12:11, C=C3=A9dric Champeau wrote:<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
Hi guys,<br><br>
After spending some days at Devoxx, I had a few chats about Jigsaw and<br>
the implications on Groovy. I will send an email summarizing them and I<br>
think we need to come up with a plan.<br></blockquote>
<br></span>
I was saying that, yes. Seems nobody wanted to believe me<span><br>
<br>
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1p=
x #ccc solid;padding-left:1ex">
I'm not sure JIRA is the best way<br>
to track this, but in short, we have serious issues coming, and breaking<br=
>
changes. Maybe it's a chance for us to make all the breaking changes we<br>
thought for so long.<br></blockquote>
<br></span>
I doubt it, we don't have the manpower, but we can do a bit of course<br><b=
r>
bye Jochen<br><br><br></blockquote></div><br></div></div></div></blockquote=
><div><br></div></div></blockquote></div><br></div></div></blockquote><div>=
<br></div></div></body></html>
------=_Part_49710_1234926740.1448542714104--