beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ib...@apache.org
Subject [beam] branch master updated: [BEAM-9689] Add Go quickstart for Spark runner.
Date Tue, 09 Mar 2021 22:28:02 GMT
This is an automated email from the ASF dual-hosted git repository.

ibzib pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git


The following commit(s) were added to refs/heads/master by this push:
     new d587d42  [BEAM-9689] Add Go quickstart for Spark runner.
     new d772aea  Merge pull request #14177 from ibzib/BEAM-9689
d587d42 is described below

commit d587d42abebaf635da0d072f2191c6bf9781ed92
Author: Kyle Weaver <kcweaver@google.com>
AuthorDate: Tue Mar 9 13:56:55 2021 -0800

    [BEAM-9689] Add Go quickstart for Spark runner.
---
 website/www/site/content/en/get-started/quickstart-go.md | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/website/www/site/content/en/get-started/quickstart-go.md b/website/www/site/content/en/get-started/quickstart-go.md
index c66067e..f92cc6e 100644
--- a/website/www/site/content/en/get-started/quickstart-go.md
+++ b/website/www/site/content/en/get-started/quickstart-go.md
@@ -69,8 +69,17 @@ $ wordcount --input gs://dataflow-samples/shakespeare/kinglear.txt \
             --worker_harness_container_image=apache/beam_go_sdk:latest
 {{< /highlight >}}
 
-{{< highlight class="runner-nemo" >}}
-This runner is not yet available for the Go SDK.
+{{< highlight class="runner-spark" >}}
+# Build and run the Spark job server from Beam source.
+# -PsparkMasterUrl is optional. If it is unset the job will be run inside an embedded Spark
cluster.
+$ ./gradlew :runners:spark:job-server:runShadow -PsparkMasterUrl=spark://localhost:7077
+
+# In a separate terminal, run:
+$ go install github.com/apache/beam/sdks/go/examples/wordcount
+$ wordcount --input <PATH_TO_INPUT_FILE> \
+            --output counts \
+            --runner spark \
+            --endpoint localhost:8099
 {{< /highlight >}}
 
 ## Next Steps


Mime
View raw message