beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dumi Loghin <dumilog...@gmail.com>
Subject Re: Flink Wordcount example on single node "No runner was specified"
Date Wed, 27 Jul 2016 05:05:52 GMT
Hi,

The application is running if the input file is in the examples folder:

~/incubator-beam/runners/flink/examples$
/home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
-Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
*-Dinput=kinglear.txt* -Doutput=wc-out.txt

( as opposed to "~/incubator-beam/runners/flink/examples$
/home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
-Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
*-Dinput=/home/hadoop/kinglear.txt* -Doutput=wc-out.txt" that is throwing
the exception)

Best regards,
Dumi



On Tue, Jul 26, 2016 at 11:53 PM, Maximilian Michels <mxm@apache.org> wrote:

> Hi Dumi,
>
> The execution fails during shutdown of the local Flink cluster. Does
> the program execute before the exception occurs?
>
> Best,
> Max
>
> On Tue, Jul 26, 2016 at 5:15 AM, Dumi Loghin <dumiloghin@gmail.com> wrote:
> > Hi,
> >
> > Thank you!
> >
> > I have manually added WordCount.java and pom.xml to the latest Beam code
> and
> > try to run it. But now I'm getting the following exception:
> >
> > :~/incubator-beam/runners/flink/examples$
> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> > -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
> >
> > java.lang.reflect.InvocationTargetException
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >         at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >         at java.lang.reflect.Method.invoke(Method.java:606)
> >         at
> org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> >         at java.lang.Thread.run(Thread.java:745)
> > Caused by: java.lang.RuntimeException: Pipeline execution failed
> >         at
> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
> >         at
> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
> >         at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
> >         at
> > org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
> >         ... 6 more
> > Caused by: java.util.concurrent.TimeoutException: Futures timed out after
> > [10000 milliseconds]
> >         at
> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> >         at
> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
> >         at
> scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
> >         at
> scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
> >         at
> >
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> >         at scala.concurrent.Await$.ready(package.scala:86)
> >         at
> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
> >         at
> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
> >         at
> > org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
> >         at
> > org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
> >         at
> >
> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
> >         at
> >
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
> >         at
> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
> >         ... 9 more
> >
> > Best regards,
> > Dumi
> >
> > On Mon, Jul 25, 2016 at 7:14 PM, Maximilian Michels <mxm@apache.org>
> wrote:
> >>
> >> Hi Dumi,
> >>
> >> Somewhere on the way we broke the exec configuration. I've fixed this
> >> in this PR and updated the README:
> >> https://github.com/apache/incubator-beam/pull/724
> >>
> >> Should be merged soon.
> >>
> >> Best,
> >> Max
> >>
> >> On Fri, Jul 22, 2016 at 2:49 PM, Lukasz Cwik <lcwik@google.com> wrote:
> >> > I think runner should be org.apache.beam.runners.flink.FlinkRunner and
> >> > not
> >> > org.apache.beam.runners.flink.FlinkRunner.class but that is probably
> >> > unrelated to the failure that you are seeing.
> >> >
> >> > On Fri, Jul 22, 2016 at 7:42 AM, Dumi Loghin <dumiloghin@gmail.com>
> >> > wrote:
> >> >>
> >> >> Hi,
> >> >>
> >> >> I'm still getting an exception related to arguments. I mention that
> I'm
> >> >> working with commit adb472083460bc175a0685133eadb6a336f1a4a2 (22
> Jul).
> >> >>
> >> >> ~/incubator-beam/runners/flink/examples$ mvn exec:java
> >> >> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> >> >> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> >> >> --inputFile=/home/hadoop/kinglear.txt --output=wc-out.txt"
> >> >> ...
> >> >> [ERROR] Failed to execute goal
> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on
> project
> >> >> beam-runners-flink_2.10-examples: Unable to parse configuration of
> mojo
> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java for parameter
> arguments:
> >> >> Cannot store value into array: ArrayStoreException -> [Help 1]
> >> >> ...
> >> >>
> >> >> However, I was able to run it in early June. If I revert to, for
> >> >> example,
> >> >> commit c2146b9f9d6a1f39a5699725ccb51829d751b88a (6 Jun) I'm not
> getting
> >> >> the
> >> >> above exceptions. Maybe something changed in the options (arguments)
> >> >> handling code?
> >> >>
> >> >> Best regards,
> >> >> Dumi
> >> >>
> >> >> On Wed, Jul 20, 2016 at 7:35 PM, Aljoscha Krettek <
> aljoscha@apache.org>
> >> >> wrote:
> >> >>>
> >> >>> I think it should work like this:
> >> >>>
> >> >>> mvn exec:java \
> >> >>> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
\
> >> >>>
> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> >> >>> --inputFile='"$input"' --output='"$outfile_prefix"'"'
> >> >>>
> >> >>> On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <dumiloghin@gmail.com>
> wrote:
> >> >>>>
> >> >>>> Hi,
> >> >>>>
> >> >>>> I'm trying to run Wordcount example on Flink single node but
I'm
> >> >>>> getting
> >> >>>> this error:
> >> >>>>
> >> >>>> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
> >> >>>> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
> >> >>>> -Drunner=FlinkRunner
> >> >>>> ...
> >> >>>> Exception in thread "main" java.lang.IllegalArgumentException:
No
> >> >>>> Runner
> >> >>>> was specified and the DirectRunner was not found on the classpath.
> >> >>>> Specify a runner by either:
> >> >>>>     Explicitly specifying a runner by providing the 'runner'
> property
> >> >>>>     Adding the DirectRunner to the classpath
> >> >>>>     Calling 'PipelineOptions.setRunner(PipelineRunner)' directly
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
> >> >>>> at
> >> >>>>
> >> >>>>
> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
> >> >>>> ...
> >> >>>>
> >> >>>>
> >> >>>> I've tried with -Drunner=FlinkRunner and without it and I get
the
> >> >>>> same
> >> >>>> error. Moreover, the runner is also set in the source:
> >> >>>>
> >> >>>> options.setRunner(FlinkRunner.class);
> >> >>>>
> >> >>>> Any suggestions?
> >> >>>>
> >> >>>> Best regards,
> >> >>>> Dumi
> >> >>
> >> >>
> >> >
> >
> >
>

Mime
View raw message