spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zoltán Zvara <zoltan.zv...@gmail.com>
Subject Re: Can't assembly YARN project with SBT
Date Wed, 25 Mar 2015 11:43:47 GMT
Hi!

It seems that the problem of "unable to load YARN support" present only
when I run my job from code and by not using the spark-submit script. IMO
this is related to SPARK-5144
<https://issues.apache.org/jira/browse/SPARK-5144>. I'm running QueueStream
example with a single change: sparkConf.setMaster("yarn-client").

I'm building Spark with: sbt/sbt -Pyarn -Dhadoop.version=2.6.0 assembly

Zvara Zoltán



mail, hangout, skype: zoltan.zvara@gmail.com

mobile, viber: +36203129543

bank: 10918001-00000021-50480008

address: Hungary, 2475 Kápolnásnyék, Kossuth 6/a

elte: HSKSJZ (ZVZOAAI.ELTE)

2015-03-25 9:45 GMT+01:00 Zoltán Zvara <zoltan.zvara@gmail.com>:

> Hi!
>
> I'm using the latest IntelliJ and I can't compile the yarn project into
> the Spark assembly fat JAR. That is why I'm getting a SparkException with
> message "Unable to load YARN support". The yarn project is also missing
> from SBT tasks and I can't add it. How can I force sbt to include?
>
> Thanks!
>
> Zvara Zoltán
>
>
>
> mail, hangout, skype: zoltan.zvara@gmail.com
>
> mobile, viber: +36203129543
>
> bank: 10918001-00000021-50480008
>
> address: Hungary, 2475 Kápolnásnyék, Kossuth 6/a
>
> elte: HSKSJZ (ZVZOAAI.ELTE)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message