spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Davies Liu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (SPARK-10436) spark-submit overwrites spark.files defaults with the job script filename
Date Thu, 03 Sep 2015 20:49:46 GMT

     [ https://issues.apache.org/jira/browse/SPARK-10436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Davies Liu updated SPARK-10436:
-------------------------------
    Target Version/s: 1.6.0

> spark-submit overwrites spark.files defaults with the job script filename
> -------------------------------------------------------------------------
>
>                 Key: SPARK-10436
>                 URL: https://issues.apache.org/jira/browse/SPARK-10436
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Submit
>    Affects Versions: 1.4.0
>         Environment: Ubuntu, Spark 1.4.0 Standalone
>            Reporter: axel dahl
>            Priority: Minor
>              Labels: easyfix, feature
>
> In my spark-defaults.conf I have configured a set of libararies to be uploaded to my
Spark 1.4.0 Standalone cluster.  The entry appears as:
> spark.files              libarary.zip,file1.py,file2.py
> When I execute spark-submit -v test.py
> I see that spark-submit reads the defaults correctly, but that it overwrites the "spark.files"
default entry and replaces it with the name if the job script, i.e. "test.py".
> This behavior doesn't seem intuitive.  test.py, should be added to the spark working
folder, but it should not overwrite the "spark.files" defaults.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message