spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Wendell (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-1520) Spark assembly fails with Java 6
Date Thu, 17 Apr 2014 06:50:16 GMT

     [ https://issues.apache.org/jira/browse/SPARK-1520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Patrick Wendell updated SPARK-1520:
-----------------------------------

    Description: 
This is a real doozie - when compiling a Spark assembly with JDK7, the produced jar does not
work well with JRE6. I confirmed the byte code being produced is JDK 6 compatible (major version
50). What happens is that, silently, the JRE will not load any class files from the assembled
jar.

{code}
$> sbt/sbt assembly/assembly

$> /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -cp /home/patrick/Documents/spark/assembly/target/scala-2.10/spark-assembly-1.0.0-SNAPSHOT-hadoop1.0.4.jar
org.apache.spark.ui.UIWorkloadGenerator
usage: ./bin/spark-class org.apache.spark.ui.UIWorkloadGenerator [master] [FIFO|FAIR]

$> /usr/lib/jvm/java-1.6.0-openjdk-amd64/bin/java -cp /home/patrick/Documents/spark/assembly/target/scala-2.10/spark-assembly-1.0.0-SNAPSHOT-hadoop1.0.4.jar
org.apache.spark.ui.UIWorkloadGenerator
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/ui/UIWorkloadGenerator
Caused by: java.lang.ClassNotFoundException: org.apache.spark.ui.UIWorkloadGenerator
	at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.spark.ui.UIWorkloadGenerator. Program will exit.

{code}

I also noticed that if the jar is unzipped, and the classpath set to the currently directory,
it "just works". Finally, if the assembly jar is compiled with JDK6, it also works.

  was:
This is a real doozie - when compiling a Spark assembly with JDK7, the produced jar does not
work well with JRE6. I confirmed the byte code being produced is JDK 6 compatible (major version
50). What happens is that, silently, the JRE will not load any class files from the assembled
jar.

{code}
patrick@patrick-t430s: sbt/sbt assembly/assembly
patrick@patrick-t430s:~/Documents/spark$ /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -cp
/home/patrick/Documents/spark/assembly/target/scala-2.10/spark-assembly-1.0.0-SNAPSHOT-hadoop1.0.4.jar
org.apache.spark.ui.UIWorkloadGenerator
usage: ./bin/spark-class org.apache.spark.ui.UIWorkloadGenerator [master] [FIFO|FAIR]
patrick@patrick-t430s:~/Documents/spark$ /usr/lib/jvm/java-1.6.0-openjdk-amd64/bin/java -cp
/home/patrick/Documents/spark/assembly/target/scala-2.10/spark-assembly-1.0.0-SNAPSHOT-hadoop1.0.4.jar
org.apache.spark.ui.UIWorkloadGenerator
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/ui/UIWorkloadGenerator
Caused by: java.lang.ClassNotFoundException: org.apache.spark.ui.UIWorkloadGenerator
	at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: org.apache.spark.ui.UIWorkloadGenerator. Program will exit.

{code}

I also noticed that if the jar is unzipped, and the classpath set to the currently directory,
it "just works". Finally, if the assembly jar is compiled with JDK6, it also works.


> Spark assembly fails with Java 6
> --------------------------------
>
>                 Key: SPARK-1520
>                 URL: https://issues.apache.org/jira/browse/SPARK-1520
>             Project: Spark
>          Issue Type: Bug
>            Reporter: Patrick Wendell
>            Priority: Blocker
>             Fix For: 1.0.0
>
>
> This is a real doozie - when compiling a Spark assembly with JDK7, the produced jar does
not work well with JRE6. I confirmed the byte code being produced is JDK 6 compatible (major
version 50). What happens is that, silently, the JRE will not load any class files from the
assembled jar.
> {code}
> $> sbt/sbt assembly/assembly
> $> /usr/lib/jvm/java-1.7.0-openjdk-amd64/bin/java -cp /home/patrick/Documents/spark/assembly/target/scala-2.10/spark-assembly-1.0.0-SNAPSHOT-hadoop1.0.4.jar
org.apache.spark.ui.UIWorkloadGenerator
> usage: ./bin/spark-class org.apache.spark.ui.UIWorkloadGenerator [master] [FIFO|FAIR]
> $> /usr/lib/jvm/java-1.6.0-openjdk-amd64/bin/java -cp /home/patrick/Documents/spark/assembly/target/scala-2.10/spark-assembly-1.0.0-SNAPSHOT-hadoop1.0.4.jar
org.apache.spark.ui.UIWorkloadGenerator
> Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/ui/UIWorkloadGenerator
> Caused by: java.lang.ClassNotFoundException: org.apache.spark.ui.UIWorkloadGenerator
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
> Could not find the main class: org.apache.spark.ui.UIWorkloadGenerator. Program will
exit.
> {code}
> I also noticed that if the jar is unzipped, and the classpath set to the currently directory,
it "just works". Finally, if the assembly jar is compiled with JDK6, it also works.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message