flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sourabh Mehta <sourabhmehta2...@gmail.com>
Subject Re: Unable to run flink job in dataproc cluster with jobmanager provided
Date Wed, 17 Jun 2020 17:39:29 GMT
No, I am not.

On Wed, 17 Jun 2020 at 10:48 PM, Chesnay Schepler <chesnay@apache.org>
wrote:

> Are you by any chance creating a local environment via
> (Stream)ExecutionEnvironment#createLocalEnvironment?
>
> On 17/06/2020 17:05, Sourabh Mehta wrote:
>
> Hi Team,
>
> I'm  exploring flink for one of my use case, I'm facing some issues while
> running a flink job in cluster mode. Below are the steps I followed to
> setup and run job in cluster mode :
> 1. Setup flink on google cloud dataproc using
> https://github.com/GoogleCloudDataproc/initialization-actions/tree/master/flink
>
> 2. After setting up the cluster I could see the flink session started and
> could see the UI for the same.
>
> 3 Submitted job from dataproc master node using below command
>
> sudo HADOOP_CONF_DIR=/etc/hadoop/conf /usr/lib/flink/bin/flink run -m
> yarn-cluster -yid application_1592311654771_0001 -class
> com.sm.flink.FlinkDriver /usr/lib/flink/lib/flink-1.0.10-sm-SNAPSHOT.jar
> hdfs://cluster-flink-poc-m:8020/user/flink/rocksdb/
>
> After running the job I see the job started successfully but created a
> mini local cluster and ran in local mode. I don't see any jobs submitted to
> JobManger and I also see 0 task managers on UI.
>
> Can someone please help me understand here?, do let me know what input is
> required to investigate the same.
>
>
>
>
>

Mime
View raw message