spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guoqiang Li (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-1659) improvements spark-submit usage
Date Mon, 28 Apr 2014 16:54:22 GMT

    [ https://issues.apache.org/jira/browse/SPARK-1659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983197#comment-13983197
] 

Guoqiang Li commented on SPARK-1659:
------------------------------------

./bin/spark-submit /opt/spark/classes/toona-assembly-1.0.0-SNAPSHOT.jar  --verbose  --master
spark://spark:7077 --deploy-mode client --class com.zhe800.toona.als.computation.DealCF 20140425
 => 
{code}
Using properties file: /opt/spark/spark-1.0.0-cdh3/conf/spark-defaults.conf
Adding default property: spark.eventLog.enabled=true
Adding default property: spark.akka.askTimeout=120
Adding default property: spark.default.parallelism=32
Adding default property: spark.executor.extraJavaOptions=-Xss5m -server -XX:+UseConcMarkSweepGC
-XX:+ExplicitGCInvokesConcurrent -XX:+CMSClassUnloadingEnabled -XX:+AggressiveOpts -XX:PermSize=150M
-XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=150M
Adding default property: spark.ui.killEnabled=false
Adding default property: spark.storage.memoryFraction=0.7
Adding default property: spark.locality.wait=10000
Adding default property: spark.executor.memory=13g
Adding default property: spark.master=spark://spark:7077
Adding default property: spark.storage.blockManagerTimeoutIntervalMs=6000000
Adding default property: spark.akka.timeout=120
Adding default property: spark.akka.frameSize=1600
Adding default property: spark.broadcast.blockSize=4096
Adding default property: spark.eventLog.dir=/opt/spark/logs/
Adding default property: spark.driver.extraJavaOptions=-Xss5m -server -XX:+UseConcMarkSweepGC
-XX:+ExplicitGCInvokesConcurrent -XX:+CMSClassUnloadingEnabled -XX:+AggressiveOpts -XX:PermSize=150M
-XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=150M
Using properties file: /opt/spark/spark-1.0.0-cdh3/conf/spark-defaults.conf
Adding default property: spark.eventLog.enabled=true
Adding default property: spark.akka.askTimeout=120
Adding default property: spark.default.parallelism=32
Adding default property: spark.executor.extraJavaOptions=-Xss5m -server -XX:+UseConcMarkSweepGC
-XX:+ExplicitGCInvokesConcurrent -XX:+CMSClassUnloadingEnabled -XX:+AggressiveOpts -XX:PermSize=150M
-XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=150M
Adding default property: spark.ui.killEnabled=false
Adding default property: spark.storage.memoryFraction=0.7
Adding default property: spark.locality.wait=10000
Adding default property: spark.executor.memory=13g
Adding default property: spark.master=spark://spark:7077
Adding default property: spark.storage.blockManagerTimeoutIntervalMs=6000000
Adding default property: spark.akka.timeout=120
Adding default property: spark.akka.frameSize=1600
Adding default property: spark.broadcast.blockSize=4096
Adding default property: spark.eventLog.dir=/opt/spark/logs/
Adding default property: spark.driver.extraJavaOptions=-Xss5m -server -XX:+UseConcMarkSweepGC
-XX:+ExplicitGCInvokesConcurrent -XX:+CMSClassUnloadingEnabled -XX:+AggressiveOpts -XX:PermSize=150M
-XX:MaxPermSize=512M -XX:ReservedCodeCacheSize=150M
{code}
printed twice

> improvements spark-submit usage
> -------------------------------
>
>                 Key: SPARK-1659
>                 URL: https://issues.apache.org/jira/browse/SPARK-1659
>             Project: Spark
>          Issue Type: Sub-task
>          Components: Spark Core, YARN
>            Reporter: Guoqiang Li
>             Fix For: 1.0.0
>
>
> Delete spark-submit obsolete usage: "--arg ARG"



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message