beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aviem Zur <>
Subject Re: Spark job hangs up at Evaluating ParMultiDo(ParseInput)
Date Thu, 10 Aug 2017 07:13:33 GMT
Hi Jayaraman,

Thanks for reaching out.
We run Beam using Spark runner daily on a yarn cluster.

It appears that in many of the logs you sent there is hanging when
connecting to certain servers on certain ports, could this be a network
issue or an issue with your Spark setup?

Could you please share which version of Beam you are running?

On Thu, Aug 3, 2017 at 12:18 PM Sathish Jayaraman <>

> Hi,
> Thanks for trying it out.
> I was running the job in local single node setup. I also spawn a
> HDInsights cluster in Azure platform just to test the WordCount program.
> Its the same result there too, stuck at the Evaluating ParMultiDo step. It
> runs fine in mvn compile exec, but when bundled into jar & submitted via
> spark-submit there is no result. If there is no support from Beam to run on
> top of Spark then I have to write Spark native code which is what I am
> doing currently.
> Regards,
> Sathish. J
> On 03-Aug-2017, at 2:34 PM, Jean-Baptiste Onofré <> wrote:
> <>

View raw message