Mailing-List: contact user-help@spark.apache.org; run by ezmlm
Precedence: bulk
MIME-Version: 1.0
In-Reply-To: 
 <CAAOnQ7uvQ6ZXWpehGoE2ARYuPcUSi8wfQBf7WsjsyhbtpSp=YQ@mail.gmail.com>
References: <1444136264370-24951.post@n3.nabble.com>
	<CAAOnQ7th9FSx+Wus6TGx2_tkWy+wYwrbkGHS7_T+20vHKyvMkg@mail.gmail.com>
	<CAP1MXB6eJZ03GGHnO1yncTSTh9d2w7_XbaDLUTPBthE5bQzWiQ@mail.gmail.com>
	<CAAOnQ7uvQ6ZXWpehGoE2ARYuPcUSi8wfQBf7WsjsyhbtpSp=YQ@mail.gmail.com>
Date: Wed, 7 Oct 2015 14:23:48 -0300
Message-ID: 
 <CAP1MXB5X_jLc6yxjWey5JTMQ5r97gkuPwVFLzCuP=HdO2aTp+w@mail.gmail.com>
Subject: Re: compatibility issue with Jersey2
From: Gary Ogden <gogden@gmail.com>
To: Marcelo Vanzin <vanzin@cloudera.com>
Cc: "user@spark.apache.org" <user@spark.apache.org>
Content-Type: multipart/alternative; boundary=001a1147b2b07257a1052187023e

--001a1147b2b07257a1052187023e
Content-Type: text/plain; charset=UTF-8

What you suggested seems to have worked for unit tests. But now it throws
this at run time on mesos with spark-submit:

Exception in thread "main" java.lang.LinkageError: loader constraint
violation: when resolving method
"org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()Lorg/slf4j/ILoggerFactory;"
the class loader (instance of
org/apache/spark/util/ChildFirstURLClassLoader) of the current class,
org/slf4j/LoggerFactory, and the class loader (instance of
sun/misc/Launcher$AppClassLoader) for resolved class,
org/slf4j/impl/StaticLoggerBinder, have different Class objects for
the type LoggerFactory; used in the signature
	at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:336)
	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:284)
	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:305)
	at com.company.spark.utils.SparkJob.<clinit>(SparkJob.java:41)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Unknown Source)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:634)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)


On 6 October 2015 at 16:20, Marcelo Vanzin <vanzin@cloudera.com> wrote:

> On Tue, Oct 6, 2015 at 12:04 PM, Gary Ogden <gogden@gmail.com> wrote:
> > But we run unit tests differently in our build environment, which is
> > throwing the error. It's setup like this:
> >
> > I suspect this is what you were referring to when you said I have a
> problem?
>
> Yes, that is what I was referring to. But, in your test environment,
> you might be able to work around the problem by setting
> "spark.ui.enabled=false"; that should disable all the code that uses
> Jersey, so you can use your newer version in your unit tests.
>
>
> --
> Marcelo
>

--001a1147b2b07257a1052187023e
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">What you suggested seems to have worked for unit tests. Bu=
t now it throws this at run time on mesos with spark-submit:<div><br><pre i=
d=3D"data" style=3D"color:rgb(0,0,0);font-size:14px;line-height:16px;width:=
1725px;height:880px;overflow:auto">Exception in thread &quot;main&quot; jav=
a.lang.LinkageError: loader constraint violation: when resolving method &qu=
ot;org.slf4j.impl.StaticLoggerBinder.getLoggerFactory()Lorg/slf4j/ILoggerFa=
ctory;&quot; the class loader (instance of org/apache/spark/util/ChildFirst=
URLClassLoader) of the current class, org/slf4j/LoggerFactory, and the clas=
s loader (instance of sun/misc/Launcher$AppClassLoader) for resolved class,=
 org/slf4j/impl/StaticLoggerBinder, have different Class objects for the ty=
pe LoggerFactory; used in the signature
	at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:336)
	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:284)
	at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:305)
	at com.company.spark.utils.SparkJob.&lt;clinit&gt;(SparkJob.java:41)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Unknown Source)
	at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmi=
t$$runMain(SparkSubmit.scala:634)
	at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:170)
	at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:193)
	at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:112)
	at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)</pre></div>=
</div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On 6 Octobe=
r 2015 at 16:20, Marcelo Vanzin <span dir=3D"ltr">&lt;<a href=3D"mailto:van=
zin@cloudera.com" target=3D"_blank">vanzin@cloudera.com</a>&gt;</span> wrot=
e:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-l=
eft:1px #ccc solid;padding-left:1ex"><span class=3D"">On Tue, Oct 6, 2015 a=
t 12:04 PM, Gary Ogden &lt;<a href=3D"mailto:gogden@gmail.com">gogden@gmail=
.com</a>&gt; wrote:<br>
&gt; But we run unit tests differently in our build environment, which is<b=
r>
&gt; throwing the error. It&#39;s setup like this:<br>
&gt;<br>
</span><span class=3D"">&gt; I suspect this is what you were referring to w=
hen you said I have a problem?<br>
<br>
</span>Yes, that is what I was referring to. But, in your test environment,=
<br>
you might be able to work around the problem by setting<br>
&quot;spark.ui.enabled=3Dfalse&quot;; that should disable all the code that=
 uses<br>
Jersey, so you can use your newer version in your unit tests.<br>
<span class=3D"HOEnZb"><font color=3D"#888888"><br>
<br>
--<br>
Marcelo<br>
</font></span></blockquote></div><br></div>

--001a1147b2b07257a1052187023e--