beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sathish Jayaraman <>
Subject Re: Spark job hangs up at Evaluating ParMultiDo(ParseInput)
Date Tue, 01 Aug 2017 10:46:36 GMT
Hi JB,

Not when I submit is using the option —master local. But when I submit with with spark master’s
port, The job executes and hangs at step 'Registering block manager with
366.3 MB RAM, BlockManagerId(0,, 58956, None)’. Attached screenshots of the
HistoryUI & Dashboard. Also raised a question in Stackoverflow<>

~/spark/bin/spark-submit --class org.apache.beam.examples.WordCount --master spark://Sathish-MacBook-Pro.local:7077
target/word-count-beam-0.1.jar --runner=SparkRunner --inputFile=pom.xml --output=counts


Sathish. J

On 01-Aug-2017, at 4:02 PM, Jean-Baptiste Onofré <<>>

Hi Sathish,

Do you see the tasks submitted on the history server ?


On 08/01/2017 11:51 AM, Sathish Jayaraman wrote:
I am trying to execute Beam example in local spark setup. When I try to submit the sample
WordCount jar via spark-submit, the job just hangs at 'INFO SparkRunner$Evaluator: Evaluating
ParMultiDo(ExtractWords)’. But it runs fine when executed directly. Below is the command
I used to submit the job in spark local,
$ ~/spark/bin/spark-submit --class "org.apache.beam.examples.WordCount"   --master local[4]
target/word-count-beam-0.1.jar —inputFile=./pom.xml --output=csvout --runner=SparkRunner
Have attached log file for reference. Can anyone please help me find out whats going on?
Sathish. J

Jean-Baptiste Onofré<>
Talend -

View raw message