spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Felix Cheung <felixcheun...@hotmail.com>
Subject Re: Is Spark 2.0 master node compatible with Spark 1.5 work node?
Date Sat, 10 Sep 2016 17:30:15 GMT
You should be able to get it to work with 2.0 as uber jar.

What type cluster you are running on? YARN? And what distribution?





On Sun, Sep 4, 2016 at 8:48 PM -0700, "Holden Karau" <holden@pigscanfly.ca<mailto:holden@pigscanfly.ca>>
wrote:

You really shouldn't mix different versions of Spark between the master and worker nodes,
if your going to upgrade - upgrade all of them. Otherwise you may get very confusing failures.

On Monday, September 5, 2016, Rex X <dnsring@gmail.com<mailto:dnsring@gmail.com>>
wrote:
Wish to use the Pivot Table feature of data frame which is available since Spark 1.6. But
the spark of current cluster is version 1.5. Can we install Spark 2.0 on the master node to
work around this?

Thanks!


--
Cell : 425-233-8271
Twitter: https://twitter.com/holdenkarau


Mime
View raw message