flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7165) Flink can't deploy on hadoop 2.6.0-cdh5.5.1 yarn
Date Fri, 14 Jul 2017 13:35:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7165?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16087320#comment-16087320
] 

Stephan Ewen commented on FLINK-7165:
-------------------------------------

Did you try to build Flink against cloudera as described here? https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/building.html#vendor-specific-versions

Ultimately, just using the Hadoop jars present (approach 3) is probably the safest thing to
do.
You can build the {{1.4-SNAPSHOT}} version as described here: https://ci.apache.org/projects/flink/flink-docs-release-1.4/setup/building.html#scala-versions

> Flink can't deploy on hadoop 2.6.0-cdh5.5.1 yarn
> ------------------------------------------------
>
>                 Key: FLINK-7165
>                 URL: https://issues.apache.org/jira/browse/FLINK-7165
>             Project: Flink
>          Issue Type: Bug
>          Components: YARN
>    Affects Versions: 1.3.1, 1.4.0
>         Environment: Flink 1.3.1 and Flink 1.4-SNAPSHOT
> hadoop 2.6.0-cdh5.5.1
>            Reporter: 刘喆
>
> When run with : 
> {quote}
>  HADOOP_CONF_DIR=/etc/hadoop/conf ./bin/flink run -m yarn-cluster  -yn 300 -yjm 4192
-ytm 1124 -c fTest.StreamingJob -p 300 ./fT.jar
> {quote}
> It failed.
> Logs as below:
> {quote}
> 2017-07-12 17:49:44.774 [main] INFO  org.apache.flink.yarn.YarnApplicationMasterRunner
 - TM:remote keytab principal obtained null
> 2017-07-12 17:49:44.774 [main] INFO  org.apache.flink.yarn.YarnApplicationMasterRunner
 - TM:remote yarn conf path obtained null
> 2017-07-12 17:49:44.774 [main] INFO  org.apache.flink.yarn.YarnApplicationMasterRunner
 - TM:remote krb5 path obtained null
> 2017-07-12 17:49:44.986 [main] ERROR org.apache.flink.yarn.YarnApplicationMasterRunner
 - YARN Application Master initialization failed
> java.lang.AbstractMethodError: org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider.getProxy()Ljava/lang/Object;
>         at org.apache.hadoop.io.retry.RetryInvocationHandler.<init>(RetryInvocationHandler.java:73)
~[fT.jar:na]
>         at org.apache.hadoop.io.retry.RetryInvocationHandler.<init>(RetryInvocationHandler.java:64)
~[fT.jar:na]
>         at org.apache.hadoop.io.retry.RetryProxy.create(RetryProxy.java:58) ~[fT.jar:na]
>         at org.apache.hadoop.hdfs.NameNodeProxies.createProxy(NameNodeProxies.java:183)
~[flink-shaded-hadoop2-uber-1.4-SNAPSHOT.jar:1.4-SNAPSHOT]
>         at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:665) ~[flink-shaded-hadoop2-uber-1.4-SNAPSHOT.jar:1.4-SNAPSHOT]
>         at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:601) ~[flink-shaded-hadoop2-uber-1.4-SNAPSHOT.jar:1.4-SNAPSHOT]
>         at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:148)
~[flink-shaded-hadoop2-uber-1.4-SNAPSHOT.jar:1.4-SNAPSHOT]
>         at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2316) ~[fT.jar:na]
>         at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:90) ~[fT.jar:na]
>         at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2350) ~[fT.jar:na]
>         at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2332) ~[fT.jar:na]
>         at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:369) ~[fT.jar:na]
>         at org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) ~[fT.jar:na]
>         at org.apache.flink.yarn.Utils.createTaskExecutorContext(Utils.java:380) ~[flink-dist_2.11-1.4-SNAPSHOT.jar:1.4-SNAPSHOT]
>         at org.apache.flink.yarn.YarnApplicationMasterRunner.runApplicationMaster(YarnApplicationMasterRunner.java:318)
[flink-dist_2.11-1.4-SNAPSHOT.jar:1.4-SNAPSHOT]
>         at org.apache.flink.yarn.YarnApplicationMasterRunner$1.call(YarnApplicationMasterRunner.java:191)
[flink-dist_2.11-1.4-SNAPSHOT.jar:1.4-SNAPSHOT]
>         at org.apache.flink.yarn.YarnApplicationMasterRunner$1.call(YarnApplicationMasterRunner.java:188)
[flink-dist_2.11-1.4-SNAPSHOT.jar:1.4-SNAPSHOT]
>         at org.apache.flink.runtime.security.HadoopSecurityContext$1.run(HadoopSecurityContext.java:44)
[flink-dist_2.11-1.4-SNAPSHOT.jar:1.4-SNAPSHOT]
>         at java.security.AccessController.doPrivileged(Native Method) [na:1.8.0_112]
>         at javax.security.auth.Subject.doAs(Subject.java:422) [na:1.8.0_112]
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
[fT.jar:na]
>         at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
[flink-dist_2.11-1.4-SNAPSHOT.jar:1.4-SNAPSHOT]
>         at org.apache.flink.yarn.YarnApplicationMasterRunner.run(YarnApplicationMasterRunner.java:188)
[flink-dist_2.11-1.4-SNAPSHOT.jar:1.4-SNAPSHOT]
>         at org.apache.flink.yarn.YarnApplicationMasterRunner.main(YarnApplicationMasterRunner.java:112)
[flink-dist_2.11-1.4-SNAPSHOT.jar:1.4-SNAPSHOT]
> {quote}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message