flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: flink performance
Date Tue, 09 Sep 2014 14:11:32 GMT
I guess in the core-site.xml, the property "fs.defaultFS" has not been set
to the namenode.
You basically put the default address there, like: "hdfs://master:8037/".

On Tue, Sep 9, 2014 at 4:08 PM, Fabian Hueske <fhueske@apache.org> wrote:

> Try to specify the HDFS path as follows:
>
> hdfs://namenode-host:namenode-port/foo/bar/
>
> Best, Fabian
>
> 2014-09-09 16:03 GMT+02:00 normanSp <wir12kqe@studserv.uni-leipzig.de>:
>
> > of course, sorry.
> >
> > command: ./bin/flink run
> > ./examples/flink-java-examples-0.7-incubating-SNAPSHOT-WordCount.jar
> > hdfs:/input.xml hdfs:/result
> >
> > org.apache.flink.client.program.ProgramInvocationException: The program
> > execution failed: java.io.IOException: The given HDFS file URI
> > (hdfs:/input.xml) did not describe the HDFS Namenode. The attempt to use
> a
> > default HDFS configuration, as specified in the 'fs.hdfs.hdfsdefault' or
> > 'fs.hdfs.hdfssite' config parameter failed due to the following problem:
> > Either no default hdfs configuration was registered, or the provided
> > configuration contains no valid hdfs namenode address (fs.default.name
> or
> > fs.defaultFS) describing the hdfs namenode host and port.
> >         at
> >
> >
> org.apache.flink.runtime.fs.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:279)
> >         at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:248)
> >         at org.apache.flink.core.fs.Path.getFileSystem(Path.java:299)
> >         at
> >
> >
> org.apache.flink.api.common.io.FileInputFormat.createInputSplits(FileInputFormat.java:391)
> >         at
> >
> >
> org.apache.flink.api.common.io.FileInputFormat.createInputSplits(FileInputFormat.java:52)
> >         at
> >
> >
> org.apache.flink.runtime.jobgraph.JobInputVertex.getInputSplits(JobInputVertex.java:101)
> >         at
> >
> >
> org.apache.flink.runtime.executiongraph.ExecutionGraph.createVertex(ExecutionGraph.java:495)
> >         at
> >
> >
> org.apache.flink.runtime.executiongraph.ExecutionGraph.constructExecutionGraph(ExecutionGraph.java:281)
> >         at
> >
> >
> org.apache.flink.runtime.executiongraph.ExecutionGraph.<init>(ExecutionGraph.java:177)
> >         at
> >
> >
> org.apache.flink.runtime.jobmanager.JobManager.submitJob(JobManager.java:477)
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >         at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:606)
> >         at org.apache.flink.runtime.ipc.RPC$Server.call(RPC.java:422)
> >         at
> org.apache.flink.runtime.ipc.Server$Handler.run(Server.java:958)
> >
> >         at org.apache.flink.client.program.Client.run(Client.java:325)
> >         at org.apache.flink.client.program.Client.run(Client.java:291)
> >         at org.apache.flink.client.program.Client.run(Client.java:285)
> >         at
> >
> >
> org.apache.flink.client.program.ContextEnvironment.execute(ContextEnvironment.java:54)
> >         at
> > org.apache.flink.example.java.wordcount.WordCount.main(WordCount.java:82)
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >         at
> >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:606)
> >         at
> >
> >
> org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:389)
> >         at
> >
> >
> org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:307)
> >         at org.apache.flink.client.program.Client.run(Client.java:244)
> >         at
> > org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:347)
> >         at org.apache.flink.client.CliFrontend.run(CliFrontend.java:334)
> >         at
> >
> org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:1001)
> >         at
> org.apache.flink.client.CliFrontend.main(CliFrontend.java:1025)
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-flink-incubator-mailing-list-archive.1008284.n3.nabble.com/flink-performance-tp1726p1746.html
> > Sent from the Apache Flink (Incubator) Mailing List archive. mailing list
> > archive at Nabble.com.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message