spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Owen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-19675) ExecutorClassLoader loads classes from SystemClassLoader
Date Wed, 22 Feb 2017 01:45:44 GMT

    [ https://issues.apache.org/jira/browse/SPARK-19675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877239#comment-15877239
] 

Sean Owen commented on SPARK-19675:
-----------------------------------

No, this isn't even a supported way to run an executor. None of this contemplates what it
would change or break in actual deployments like on YARN. As I've indicated twice this is
not a problem for Spark. At least none is argues here. 

> ExecutorClassLoader loads classes from SystemClassLoader
> --------------------------------------------------------
>
>                 Key: SPARK-19675
>                 URL: https://issues.apache.org/jira/browse/SPARK-19675
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 2.1.0, 2.2.0
>         Environment: sbt / Play Framework
>            Reporter: Kohki Nishio
>            Priority: Minor
>
> Spark Executor loads classes from SystemClassLoader which contains sbt-launch.jar and
it contains Scala2.10 binary, however Spark itself is built on Scala2.11, thus it's throwing
InvalidClassException
> java.io.InvalidClassException: scala.Option; local class incompatible: stream classdesc
serialVersionUID = -114498752079829388, local class serialVersionUID = 5081326844987135632
> 	at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:616)
> 	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630)
> 	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
> 	at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630)
> 	at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
> 	at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
> 	at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
> 	at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
> 	at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
> ExecutorClassLoader's desired class loder (parentLoader) actually contains the correct
path (scala-library-2.11.8.jar) but it is not being used.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message