spark-reviews mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From tdas <>
Subject [GitHub] spark pull request: Small updates to Streaming Programming Guide
Date Wed, 21 May 2014 01:15:01 GMT
Github user tdas commented on a diff in the pull request:
    --- Diff: docs/ ---
    @@ -355,21 +358,21 @@ object has to be created, which is the main entry point of all Spark
Streaming f
     A `JavaStreamingContext` object can be created by using
     {% highlight scala %}
    -new JavaStreamingContext(master, appName, batchInterval, [sparkHome], [jars])
    +new JavaStreamingContext(master, appName, batchDuration, [sparkHome], [jars])
     {% endhighlight %}
     The `master` parameter is a standard [Spark cluster URL](scala-programming-guide.html#master-urls)
    -and can be "local" for local testing. The `appName` is a name of your program,
    -which will be shown on your cluster's web UI. The `batchInterval` is the size of the
    +and can be `local` for local testing. The `appName` is a name of your program,
    --- End diff --
    I just realized that saying "local" is a bad idea because "local" ends up having only
one slot, which is bad for Spark Streaming (as receiver takes up one core leaving none for
computation). Can you replace this `local` with `local[*]` (which detects the number of cores
in the local system).

If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at or file a JIRA ticket
with INFRA.

View raw message