ant-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <uschind...@apache.org>
Subject Multi-Release JAR file patch as applied to build 108 of Java 9 breaks almost every project out there (Apache Ant, Gradle, partly Apache Maven)
Date Sat, 05 Mar 2016 13:24:37 GMT
Hi OpenJDK Core Developers,

you may know the Apache Lucene team is testing early access releases of Java 9. We reported
many bugs already, but most of them only applied to Hotspot and Lucene itsself. But this problem
since build 108 is now really severe, because it breaks the build system already!

To allow further testing of Open Source Projects, I'd suggest to revert the Multi-Release-JAR
runtime support patch and provide a new preview build ASAP, because we found out after a night
of debugging a build system from which we don't know all internals what is causing the problems
and there is no workaround. I am very sorry that I have to say this, but it unfortunately
build 108 breaks *ALL* versions of Apache Ant, the grandfather of all Java build systems :-)
I know also OpenJDK is using it, too! So with Multi-Release JAR file patch applied (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c),
any Ant-based build - including the JDK build itsself - would no longer bootstrap. It is impossible
to also build Gradle projects, because Gradle uses Ant internally for many tasks). Maven projects
may be affected, too.

Now you might have the question: What happened?

We tried to build Lucene on our Jenkins server, but the build itsself failed with a stupid
error message:

BUILD FAILED
/home/jenkins/workspace/Lucene-Solr-master-Linux/build.xml:21: The following error occurred
while executing this line:
/home/jenkins/workspace/Lucene-Solr-master-Linux/lucene/common-build.xml:56: not doesn't support
the nested "matches" element.

The first idea was: Ah, there were changes in XML parsing (JDK-8149915). So we debugged the
build. But it was quite clear that XML parsing was not the issue. It got quite clear when
we enabled "-debug" on the build. What happened was that Ant was not loading its internal
conditions/tasks/type definitions anymore, so the build system does not know almost any type
anymore. The debug log showed that Ant was no longer able to load the resource "/org/apache/tools/ant/antlib.xml"
from its own JAR file anymore. Instead it printed some strange debugging output (which looked
totally broken).

I spend the whole night digging through their code and found the issue: The commit of Multi-Release-Jar
files (see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c) broke resource handling
in Apache Ant. In short: If you call ClassLoader.getResources() / or getResource() you get
back an URL from where you can load the Resource - this is all fine and still works. But,
with the Multi-Release JAR files patch this now has an URL fragment appended to the URL: '#release'
(see http://hg.openjdk.java.net/jdk9/jdk9/jdk/rev/f9913ea0f95c); this also applies to non-multi-release
JAR files like Apache Ant's "ant.jar".

In Java 7, Java 8,... and Java 9pre-b108, ClassLoader.getResource()/getResources() returned
stuff like: 

"jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml"

Now in Java 9b108 the following is returned:

"jar:file:/C:/Program%20Files/Java/apache-ant-1.9.6/lib/ant.jar!/org/apache/tools/ant/antlib.xml#release"

And here Ant breaks (and I assume many other projects like Maven, too). Ant checks for the
file extension of the string (because it may load definitions from both XML and properties
files). So it does endsWith(".xml") and of course this now returns false. The effect is that
Ant tries to load its own task definitions as a java properties file instead of XML. Of course
this fails, because the data behind this URL is XML. The effect is that Ant cannot bootstrap
as everything to build is missing.

One might say: Ant's code is broken (I agree, it is not nice because it relies on the string
representation of the resource URL - which is a no-go anyways), but it is impossible to fix,
because Ant is bundled on most developer computers and those will suddenly break with Java
9! There is also no version out there that works around this, so we cannot test anything anymore!

The problematic line in Ant's code is here: http://grepcode.com/file/repo1.maven.org/maven2/org.apache.ant/ant/1.9.6/org/apache/tools/ant/taskdefs/Definer.java?av=f#259

I'd suggest to please ASAP revert the Multi-Release JAR file patch and provide a new preview
build as soon as possible. I think there is more work needed to fix this. If this does not
revert to the original state, it will be impossible to build and test Lucene, Elasticsearch,....
(and almost every Java project out there!). So short: We cannot test anymore and it is likely
that we cannot support Java 9 anymore because the build system used by most Java projects
behind the scenes does not bootstrap itself anymore.

My suggestion would be to investigate other versions for this patch that does *not* modify
the resource URLs by appending a fragment to them (at least not for the "standard" case without
an actual Multi-Release Jar). For new multi-release JAR files I am fine with appending fragments,
but please not for default ones. Maybe change code to handle the URLs from the non-versioned
part differently (without fragment). Leaving the fragment inide may break many othe rprojects,
because many programmers are very sloppy with handling URLs (well-known issue is calling URL#getFile()
of a file:-URL that breaks on Windows systems and spaces in path name). Many people just call
toString() on URL and do some test on it (startsWith, endsWith). So appending fragments is
a no-go for backwards compatibility with JAR resources!

I posted this to the mailing list and did not open a bug report on http://bugs.java.com/,
because this is a more general issue - feel free to open bug reports around this!!! I would
be very happy if we could find a quick solution for this problem. Until there is a solution
we have to stop testing Java 9 with Apache Lucene/Solr/..., and this is not a good sign, especially
as Jigsaw will be merged soon.

Thanks for listening,
Uwe

P.S.: I also CCed the Apache Ant team. They should fix the broken code anyways, but this won't
help for many projects already out there (e.g. Apache Lucene still has a minimum requirement
of Ant 1.8.2 because MacOSX computers ship with that version since years).

-----
Uwe Schindler
uschindler@apache.org 
ASF Member, Apache Lucene PMC / Committer
Bremen, Germany
http://lucene.apache.org/



---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@ant.apache.org
For additional commands, e-mail: dev-help@ant.apache.org


Mime
View raw message