flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 张炜 <zhangwei...@gmail.com>
Subject link on yarn error “Yarn only has -1 virtual cores available”
Date Tue, 23 Oct 2018 04:52:11 GMT
Dear all,

I have installed Cloudera CDH 6, and then I want to install flink using
package from flink website, and run it "flink on yarn".

I have down the following steps: 1. edit /etc/profile

export
HADOOP_CONF_DIR=/opt/cloudera/parcels/CDH-6.0.0-1.cdh6.0.0.p0.537114/lib/hadoop/etc/hadoop

   1. execute ./yarn-session.sh -n 4 -jm 2048 -tm 2048 -s 3 -nm
   FlinkOnYarnSession -d -st

But it always shows:

The number of virtual cores per node were configured with 4 but Yarn only
has -1 virtual cores available. Please note that the number of virtual
cores is set to the number of task slots by default unless configured in
the Flink config with 'yarn.containers.vcores.'

This is a new cluster and there is no job running, and I see in yarn webUI
there is 20 vcores available.

Also after I tried to add yarn.containers.vcores in flink-conf.yml, say I
add a line yarn.containers.vcores: 8, when I run the yarn-session command,
it will throw an error "The number of virtual cores per node were
configured with 8 but Yarn only has -1 virtual cores available.".
[image: flink-error.png]

Please help with this problem. Thank you very much!

Mime
View raw message