flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@apache.org>
Subject Re: flink performance
Date Tue, 09 Sep 2014 14:33:00 GMT
This error message basically says that Flink's HDFS-client and the running
HDFS are not compatible.

We have different builds for Hadoop 1.0 and Hadoop 2.0.
Please check the version of the running HDFS and choose the corresponding
build.

Best, Fabian

2014-09-09 16:32 GMT+02:00 Fabian Hueske <fhueske@gmail.com>:

> This error message basically says that Flink's HDFS-client and the running
> HDFS are not compatible.
>
> We have different builds for Hadoop 1.0 and Hadoop 2.0.
> Please check the version of the running HDFS and choose the corresponding
> build.
>
> Best, Fabian
>
> 2014-09-09 16:23 GMT+02:00 normanSp <wir12kqe@studserv.uni-leipzig.de>:
>
>> thanks, but i tried all that versions before I posted here. and exactly
>> the
>> same conf-file works in 0.6.
>> @fabian
>> with full hostname and port I get this error:
>>
>> org.apache.flink.client.program.ProgramInvocationException: The program
>> execution failed: java.io.IOException: The given file URI
>> (hdfs://MASTER:9000/input.xml) described the host and port of an HDFS
>> Namenode, but the File System could not be initialized with that address:
>> Server IPC version 9 cannot communicate with client version 4
>>         at
>>
>> org.apache.flink.runtime.fs.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:305)
>>         at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:248)
>>         at org.apache.flink.core.fs.Path.getFileSystem(Path.java:299)
>>         at
>>
>> org.apache.flink.api.common.io.FileInputFormat.createInputSplits(FileInputFormat.java:391)
>>         at
>>
>> org.apache.flink.api.common.io.FileInputFormat.createInputSplits(FileInputFormat.java:52)
>>         at
>>
>> org.apache.flink.runtime.jobgraph.JobInputVertex.getInputSplits(JobInputVertex.java:101)
>>         at
>>
>> org.apache.flink.runtime.executiongraph.ExecutionGraph.createVertex(ExecutionGraph.java:495)
>>         at
>>
>> org.apache.flink.runtime.executiongraph.ExecutionGraph.constructExecutionGraph(ExecutionGraph.java:281)
>>         at
>>
>> org.apache.flink.runtime.executiongraph.ExecutionGraph.<init>(ExecutionGraph.java:177)
>>         at
>>
>> org.apache.flink.runtime.jobmanager.JobManager.submitJob(JobManager.java:477)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>         at org.apache.flink.runtime.ipc.RPC$Server.call(RPC.java:422)
>>         at
>> org.apache.flink.runtime.ipc.Server$Handler.run(Server.java:958)
>> Caused by: org.apache.hadoop.ipc.RemoteException: Server IPC version 9
>> cannot communicate with client version 4
>>         at org.apache.hadoop.ipc.Client.call(Client.java:1113)
>>         at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
>>         at com.sun.proxy.$Proxy1.getProtocolVersion(Unknown Source)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>         at
>>
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
>>         at
>>
>> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
>>         at com.sun.proxy.$Proxy1.getProtocolVersion(Unknown Source)
>>         at org.apache.hadoop.ipc.RPC.checkVersion(RPC.java:422)
>>         at
>> org.apache.hadoop.hdfs.DFSClient.createNamenode(DFSClient.java:183)
>>         at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:281)
>>         at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:245)
>>         at
>>
>> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:100)
>>         at
>>
>> org.apache.flink.runtime.fs.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:302)
>>         ... 15 more
>>
>>         at org.apache.flink.client.program.Client.run(Client.java:325)
>>         at org.apache.flink.client.program.Client.run(Client.java:291)
>>         at org.apache.flink.client.program.Client.run(Client.java:285)
>>         at
>>
>> org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:54)
>>         at
>> org.apache.flink.example.java.wordcount.WordCount.main(WordCount.java:82)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>>         at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>         at java.lang.reflect.Method.invoke(Method.java:606)
>>         at
>>
>> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:389)
>>         at
>>
>> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:307)
>>         at org.apache.flink.client.program.Client.run(Client.java:244)
>>         at
>> org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:347)
>>         at org.apache.flink.client.CliFrontend.run(CliFrontend.java:334)
>>         at
>> org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1001)
>>         at org.apache.flink.client.CliFrontend.main(CliFrontend.java:1025)
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/flink-performance-tp1726p1750.html
>> Sent from the Apache Flink (Incubator) Mailing List archive. mailing list
>> archive at Nabble.com.
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message