Mailing-List: contact user-help@spark.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@spark.apache.org
Received-SPF: neutral (nike.apache.org: local policy)
MIME-Version: 1.0
In-Reply-To: <1405579344086-10017.post@n3.nabble.com>
References: <1405576190898-10011.post@n3.nabble.com>
	<2A8C564F-9022-41A7-A9F2-1FBD761852C0@alpinenow.com>
	<1405579344086-10017.post@n3.nabble.com>
Date: Thu, 17 Jul 2014 18:32:31 -0700
Message-ID: 
 <CAMJOb8ne-rxby_U55F7hSi9VjQQaUHzejvMrPhV5Z8V2vQsN8A@mail.gmail.com>
Subject: Re: jar changed on src filesystem
From: Andrew Or <andrew@databricks.com>
To: user@spark.apache.org
Cc: "user@spark.incubator.apache.org" <user@spark.incubator.apache.org>
Content-Type: multipart/alternative; boundary=089e01227abc2b58e704fe6dbb70

--089e01227abc2b58e704fe6dbb70
Content-Type: text/plain; charset=UTF-8

Hi Jian,

In yarn-cluster mode, Spark submit automatically uploads the assembly jar
to a distributed cache that all executor containers read from, so there is
no need to manually copy the assembly jar to all nodes (or pass it through
--jars).
It seems there are two versions of the same jar in your HDFS. Try removing
all old jars from your .sparkStaging directory and try again?

Let me know if that does the job,
Andrew


2014-07-16 23:42 GMT-07:00 cmti95035 <cmti95035@gmail.com>:

> They're all the same version. Actually even without the "--jars" parameter
> it
> got the same error. Looks like it needs to copy the assembly jar for
> running
> the example jar anyway during the staging.
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/jar-changed-on-src-filesystem-tp10011p10017.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>

--089e01227abc2b58e704fe6dbb70
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hi Jian,<div><br></div><div>In yarn-cluster mode, Spark su=
bmit automatically uploads the assembly jar to a distributed cache that all=
 executor containers read from, so there is no need to manually copy the as=
sembly jar to all nodes (or pass it through --jars).</div>
<div>It seems there are two versions of the same jar in your HDFS. Try remo=
ving all old jars from your .sparkStaging directory and try again?<br></div=
><div><br></div><div>Let me know if that does the job,</div><div>Andrew</di=
v>
</div><div class=3D"gmail_extra"><br><br><div class=3D"gmail_quote">2014-07=
-16 23:42 GMT-07:00 cmti95035 <span dir=3D"ltr">&lt;<a href=3D"mailto:cmti9=
5035@gmail.com" target=3D"_blank">cmti95035@gmail.com</a>&gt;</span>:<br><b=
lockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px =
#ccc solid;padding-left:1ex">
They&#39;re all the same version. Actually even without the &quot;--jars&qu=
ot; parameter it<br>
got the same error. Looks like it needs to copy the assembly jar for runnin=
g<br>
the example jar anyway during the staging.<br>
<br>
<br>
<br>
--<br>
View this message in context: <a href=3D"http://apache-spark-user-list.1001=
560.n3.nabble.com/jar-changed-on-src-filesystem-tp10011p10017.html" target=
=3D"_blank">http://apache-spark-user-list.1001560.n3.nabble.com/jar-changed=
-on-src-filesystem-tp10011p10017.html</a><br>

<div class=3D"HOEnZb"><div class=3D"h5">Sent from the Apache Spark User Lis=
t mailing list archive at Nabble.com.<br>
</div></div></blockquote></div><br></div>

--089e01227abc2b58e704fe6dbb70--