spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elkhan Dadashov <elkhan8...@gmail.com>
Subject Is there programmatic way running Spark job on Yarn cluster without using spark-submit script ?
Date Thu, 18 Jun 2015 00:29:40 GMT
Hi all,

Is there any way running Spark job in programmatic way on Yarn cluster
without using spark-submit script ?

I cannot include Spark jars on my Java application (due o dependency
conflict and other reasons), so I'll be shipping Spark assembly uber jar
(spark-assembly-1.3.1-hadoop2.3.0.jar) to Yarn cluster, and then execute
job (Python or Java) on Yarn-cluster.

So is there any way running Spark job implemented in python file/Java class
without calling it through spark-submit script ?

Thanks.

Mime
View raw message