spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nathan Kronenfeld <>
Subject Re: Change for submitting to yarn in 1.3.1
Date Fri, 22 May 2015 02:30:51 GMT
> In researching and discussing these issues with Cloudera and others, we've
> been told that only one mechanism is supported for starting Spark jobs: the
> *spark-submit* scripts.

Is this new? We've been submitting jobs directly from a programatically
created spark context (instead of through spark-submit) from the beginning
(from 0.7.x to 1.2) - to a local cluster.

In moving to 1.3 on Yarn cluster recently, we've had no end of problems
trying to switch this over (though I think we're almost there).

Why would one want to eliminate this possibility?


View raw message