flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ken Krugler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8615) Configuring Apache Flink Local Set up with Pseudo distributed Yarn
Date Fri, 09 Feb 2018 04:54:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357916#comment-16357916

Ken Krugler commented on FLINK-8615:


> Configuring Apache Flink Local Set up with Pseudo distributed Yarn
> ------------------------------------------------------------------
>                 Key: FLINK-8615
>                 URL: https://issues.apache.org/jira/browse/FLINK-8615
>             Project: Flink
>          Issue Type: Bug
>    Affects Versions: 1.4.0
>         Environment: Mac Os, Hadoop 2.8.3 and Flink 1.4.0
>            Reporter: Karrtik Iyer
>            Priority: Major
> I have set up HADOOP 2.8.3 and YARN on my Mac machine in Pseudo distributed mode, following
this blog: [Hadoop In Pseudo Distributed Mode|http://zhongyaonan.com/hadoop-tutorial/setting-up-hadoop-2-6-on-mac-osx-yosemite.html] I
have been able to successfully start hdfs and yarn. And also able to submit Map reduce jobs.
> After that I have download Apache Flink 1.4 from [here|http://redrockdigimark.com/apachemirror/flink/flink-1.4.0/flink-1.4.0-bin-hadoop28-scala_2.11.tgz].
Now I am trying to set up Flink on the above Yarn cluster by following the steps [here|https://ci.apache.org/projects/flink/flink-docs-release-1.4/ops/deployment/yarn_setup.html].
When I try to run /bin/yarn-session.sh -n 4 -jm 1024 -tm 4096, I am getting below error which
I am unable to resolve. Can someone please advise and help me with the same? 
> {{Error while deploying YARN cluster: Couldn't deploy Yarn session cluster java.lang.RuntimeException:
Couldn't deploy Yarn session cluster at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:372)
at org.apache.flink.yarn.cli.FlinkYarnSessionCli.run(FlinkYarnSessionCli.java:679) at org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:514)
at org.apache.flink.yarn.cli.FlinkYarnSessionCli$1.call(FlinkYarnSessionCli.java:511) at java.security.AccessController.doPrivileged(Native
Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
at org.apache.flink.yarn.cli.FlinkYarnSessionCli.main(FlinkYarnSessionCli.java:511) Caused
by: org.apache.flink.configuration.IllegalConfigurationException: The number of virtual cores
per node were configured with 1 but Yarn only has -1 virtual cores available. Please note
that the number of virtual cores is set to the number of task slots by default unless configured
in the Flink config with 'yarn.containers.vcores.' at org.apache.flink.yarn.AbstractYarnClusterDescriptor.isReadyForDeployment(AbstractYarnClusterDescriptor.java:265)
at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deployInternal(AbstractYarnClusterDescriptor.java:415)
at org.apache.flink.yarn.AbstractYarnClusterDescriptor.deploySessionCluster(AbstractYarnClusterDescriptor.java:367)}}

This message was sent by Atlassian JIRA

View raw message