spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tom Graves <tgraves...@yahoo.com>
Subject Re: [jira] [Commented] (SPARK-1576) Passing of JAVA_OPTS to YARN on command line
Date Wed, 23 Apr 2014 15:51:07 GMT
can you be more specific?  What breaks existing jobs?  If you are referring to my comment,
 SPARK_JAVA_OPTS still works but I think the intent is to move away from it.

Tom
On Wednesday, April 23, 2014 8:07 AM, Mridul Muralidharan <mridul@gmail.com> wrote:
 
This breaks all existing jobs which are not using spark-submit.
The consensus was not to break compatibility unless there was an overriding
reason to do so

On Apr 23, 2014 6:32 PM, "Thomas Graves (JIRA)" <jira@apache.org> wrote:

>
>     [
> https://issues.apache.org/jira/browse/SPARK-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13978164#comment-13978164]
>
> Thomas Graves commented on SPARK-1576:
> --------------------------------------
>
> Is this meant for the driver or the executors?  The spark-submit script
> has a command line option for the driver:  --driver-java-options.
> I believe the intent of https://github.com/apache/spark/pull/299 was to
> not expose SPARK_JAVA_OPTS to the user anymore.
>
> > Passing of JAVA_OPTS to YARN on command line
> > --------------------------------------------
> >
> >                 Key: SPARK-1576
> >                 URL: https://issues.apache.org/jira/browse/SPARK-1576
> >             Project: Spark
> >          Issue Type: Improvement
> >    Affects Versions: 0.9.0, 1.0.0, 0.9.1
> >            Reporter: Nishkam Ravi
> >             Fix For: 0.9.0, 1.0.0, 0.9.1
> >
> >         Attachments: SPARK-1576.patch
> >
> >
> > JAVA_OPTS can be passed by using either env variables (i.e.,
> SPARK_JAVA_OPTS) or as config vars (after Patrick's recent change). It
> would be good to allow the user to pass them on command line as well to
> restrict scope to single application invocation.
>
>
>
> --
> This message was sent by Atlassian JIRA
> (v6.2#6252)
>
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message