spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hyukjin Kwon (JIRA)" <>
Subject [jira] [Resolved] (SPARK-21325) The shell of 'spark-submit' about '--jars' and '--fils', jars and files can be placed on local and hdfs.
Date Fri, 28 Jul 2017 04:00:06 GMT


Hyukjin Kwon resolved SPARK-21325.
    Resolution: Invalid

The affected version you set is 2.3.0 and that looks fixed in the master and the description
is not clear to me as well. I am resolving this as invalid for now.

> The shell of 'spark-submit' about '--jars' and '--fils', jars and files can be placed
on local and hdfs.
> --------------------------------------------------------------------------------------------------------
>                 Key: SPARK-21325
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: Spark Core
>    Affects Versions: 2.3.0
>            Reporter: guoxiaolongzte
>            Priority: Minor
> 1.My submit way:
> spark-submit --class cn.gxl.TestSql{color:red} --jars hdfs://nameservice:/gxl/spark-core_2.11-2.3.0-SNAPSHOT.jar,hdfs://nameservice:/gxl/zookeeper-3.4.6.jar
--files hdfs://nameservice:/gxl/value1.txt,hdfs://nameservice:/gxl/value2.txt{color} hdfs://nameservice:/gxl/spark_2.0.2_project.jar
> 2.spark-submit description´╝Ü
> --jars {color:red}JARS Comma-separated list of local jars{color} to include on the driver
and executor classpaths.
> --files{color:red} FILES Comma-separated list of files{color} to be placed in the working
directory of each executor. File paths of these files in executors can be accessed via SparkFiles.get(fileName).
> 3.Problem Description´╝Ü
> {color:red} jars and files Not only can be placed on local but also can be placed on
> The description of '' - jars '', that can only be placed on local.This is wrong
> The description of '--files' is not clear that can be placed locally or hdfs.This is
blurry. Not conducive to the developer to understand and use.*{color}
> So, this is an optimization feature that deserves to be modified.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message