spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pwendell <>
Subject [GitHub] spark pull request: SPARK-1183. Don't use "worker" to mean executo...
Date Thu, 13 Mar 2014 06:03:42 GMT
Github user pwendell commented on a diff in the pull request:
    --- Diff: yarn/common/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala
    @@ -133,11 +148,11 @@ class ClientArguments(val args: Array[String], val sparkConf: SparkConf)
           "  --class CLASS_NAME         Name of your application's main class (required)\n"
           "  --args ARGS                Arguments to be passed to your application's main
class.\n" +
           "                             Mutliple invocations are possible, each will be passed
in order.\n" +
    -      "  --num-workers NUM          Number of workers to start (Default: 2)\n" +
    -      "  --worker-cores NUM         Number of cores for the workers (Default: 1).\n"
    -      "  --master-class CLASS_NAME  Class Name for Master (Default: spark.deploy.yarn.ApplicationMaster)\n"
    -      "  --master-memory MEM        Memory for Master (e.g. 1000M, 2G) (Default: 512
Mb)\n" +
    -      "  --worker-memory MEM        Memory per Worker (e.g. 1000M, 2G) (Default: 1G)\n"
    +      "  --num-executors NUM        Number of executors to start (Default: 2)\n" +
    +      "  --executor-cores NUM       Number of cores for the executors (Default: 1).\n"
    +      "  --am-class CLASS_NAME      Class Name for application master (Default: spark.deploy.yarn.ApplicationMaster)\n"
    --- End diff --
    On this one I'm still a bit confused. @tgravescs what was the reason for making this configurable?
Are there other ApplicationMaster implementations that could be used here?

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message