maven-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <org.apache.maven.u...@io7m.com>
Subject Strange issue with generated jars
Date Tue, 12 Nov 2013 13:02:46 GMT
Hello.

I've run into a strange but easily reproduced problem with the jar files
generated by Maven. Essentially, if I generate a jar file containing a
large number of files (>= 65536, in practice), then javac becomes unable
to resolve classes from that jar file. This only occurs with jars produced
by the Maven jar plugin, and only when the number of files is large (as
demonstrated below).

An example build, using the maven exec plugin to generate a large
number (65525) of files:

http://waste.io7m.com/2013/11/12/jarbug.zip

$ mvn -C clean verify
[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] resources
[INFO] code
[INFO] jarbug
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building resources 0.1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ resources ---
[INFO] Deleting /home/m0/doc/dev/2013/11/jarbug/resources/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ resources ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build
is platform dependent!
[INFO] skip non existing resourceDirectory /home/m0/doc/dev/2013/11/jarbug/resources/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ resources ---
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform
dependent!
[INFO] Compiling 1 source file to /home/m0/doc/dev/2013/11/jarbug/resources/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ resources ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build
is platform dependent!
[INFO] skip non existing resourceDirectory /home/m0/doc/dev/2013/11/jarbug/resources/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @ resources ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ resources ---
[INFO] No tests to run.
[INFO]
[INFO] >>> exec-maven-plugin:1.2.1:java (make-sources) @ resources >>>
[INFO]
[INFO] <<< exec-maven-plugin:1.2.1:java (make-sources) @ resources <<<
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:java (make-sources) @ resources ---
Create /home/m0/doc/dev/2013/11/jarbug/resources/target/classes/com/io7m/jarbug/things/0.txt
Create /home/m0/doc/dev/2013/11/jarbug/resources/target/classes/com/io7m/jarbug/things/1.txt
Create /home/m0/doc/dev/2013/11/jarbug/resources/target/classes/com/io7m/jarbug/things/2.txt
...
Create /home/m0/doc/dev/2013/11/jarbug/resources/target/classes/com/io7m/jarbug/things/65522.txt
Create /home/m0/doc/dev/2013/11/jarbug/resources/target/classes/com/io7m/jarbug/things/65523.txt
Create /home/m0/doc/dev/2013/11/jarbug/resources/target/classes/com/io7m/jarbug/things/65524.txt
[INFO]
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ resources ---
[INFO] Building jar: /home/m0/doc/dev/2013/11/jarbug/resources/target/resources-0.1.0.jar
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building code 0.1.0
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ code ---
[INFO] Deleting /home/m0/doc/dev/2013/11/jarbug/code/target
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ code ---
[WARNING] Using platform encoding (UTF-8 actually) to copy filtered resources, i.e. build
is platform dependent!
[INFO] skip non existing resourceDirectory /home/m0/doc/dev/2013/11/jarbug/code/src/main/resources
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @ code ---
[WARNING] File encoding has not been set, using platform encoding UTF-8, i.e. build is platform
dependent!
[INFO] Compiling 1 source file to /home/m0/doc/dev/2013/11/jarbug/code/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/m0/doc/dev/2013/11/jarbug/code/src/main/java/com/io7m/jarbug/Main.java:[8,23]
error: cannot find symbol
[INFO] 1 error
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] resources ......................................... SUCCESS [30.178s]
[INFO] code .............................................. FAILURE [0.062s]
[INFO] jarbug ............................................ SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 30.356s
[INFO] Finished at: Tue Nov 12 12:32:47 UTC 2013
[INFO] Final Memory: 14M/274M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:2.5.1:compile
(default-compile) on project code: Compilation failure
[ERROR] /home/m0/doc/dev/2013/11/jarbug/code/src/main/java/com/io7m/jarbug/Main.java:[8,23]
error: cannot find symbol
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following
articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :code

From the command line:

$ cat code/src/main/java/com/io7m/jarbug/Main.java
package com.io7m.jarbug;

public final class Main
{
  public static void main(
    final String args[])
  {
    System.out.println(MakeSources.class);
  }
}

$ javac -classpath resources/target/resources-0.1.0.jar code/src/main/java/com/io7m/jarbug/Main.java

code/src/main/java/com/io7m/jarbug/Main.java:8: error: cannot find symbol
    System.out.println(MakeSources.class);
                       ^
  symbol:   class MakeSources
  location: class Main
1 error

$ unzip -l resources/target/resources-0.1.0.jar | grep MakeSources
     1489  2013-11-12 12:32   com/io7m/jarbug/MakeSources.class
$ unzip -l resources/target/resources-0.1.0.jar | wc -l
65542

The numbers given above are actually five more than the number of files, because the
unzip command line tool produces five extra lines in the output. So resources-0.1.0.jar
really contains 65537 files.

However...

$ pushd resources/target/classes
$ javac ../../../code/src/main/java/com/io7m/jarbug/Main.java 

So it works fine if the classes and files are not in a jar file. Also:

$ pushd resources/target/classes
$ zip -r ../../../new.zip .
$ popd
$ javac -classpath new.zip code/src/main/java/com/io7m/jarbug/Main.java

Although the new.zip has fewer files (65530) than the resources-0.1.0.jar, because the 
various metadata files that Maven inserts upon jar creation is not present:

$ unzip -l new.zip | wc -l
65535

Increasing the number of files by 10, and recreating new.zip:

$ rm new.zip
$ mvn -C clean verify
...
Create /home/m0/doc/dev/2013/11/jarbug/resources/target/classes/com/io7m/jarbug/things/65532.txt
Create /home/m0/doc/dev/2013/11/jarbug/resources/target/classes/com/io7m/jarbug/things/65533.txt
Create /home/m0/doc/dev/2013/11/jarbug/resources/target/classes/com/io7m/jarbug/things/65534.txt
...

$ pushd resources/target/classes
$ zip -r ../../../new.zip .
$ popd
$ unzip -l new.zip | wc -l
65545

$ javac -classpath new.zip code/src/main/java/com/io7m/jarbug/Main.java

No error is reported, the program compiles correctly. Let's see if the 'jar' program
can produce a working jar file:

$ jar cf new.jar -C resources/target/classes .
$ unzip -l new.jar | wc -l
65547
$ javac -classpath new.jar code/src/main/java/com/io7m/jarbug/Main.java

Again, no issues. So what is it that the Maven jar plugin is doing that's producing
apparently broken jar files?

$ unzip -l new.jar | awk '{print $NF}' > new.jar.list
$ unzip -l resources/target/resources-0.1.0.jar | awk '{print $NF}' > resources.jar.list

$ diff -u new.jar.list resources.jar.list 
--- new.jar.list	2013-11-12 12:58:16.491221126 +0000
+++ resources.jar.list	2013-11-12 12:58:39.159215571 +0000
@@ -1,4 +1,4 @@
-new.jar
+resources/target/resources-0.1.0.jar
 Name
 ----
 META-INF/
@@ -65543,5 +65543,10 @@
 com/io7m/jarbug/things/8006.txt
 com/io7m/jarbug/things/4304.txt
 com/io7m/jarbug/MakeSources.class
+META-INF/maven/
+META-INF/maven/com.io7m.jarbug/
+META-INF/maven/com.io7m.jarbug/resources/
+META-INF/maven/com.io7m.jarbug/resources/pom.xml
+META-INF/maven/com.io7m.jarbug/resources/pom.properties
 -------
 files

So the only thing that really differs is that the Maven jar contains maven metadata
in the META-INF directory.

If anyone has any suggestions as to why this should cause the above insane behaviour,
I'd be glad to hear them!

M

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@maven.apache.org
For additional commands, e-mail: users-help@maven.apache.org


Mime
View raw message