beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dumi Loghin <dumilog...@gmail.com>
Subject Re: Flink Wordcount example on single node "No runner was specified"
Date Thu, 28 Jul 2016 04:50:00 GMT
Hi,

Here is the entire output of the command (when running mvn -e):

~/incubator-beam/runners/flink/examples$
/home/hadoop/apache-maven-3.3.9/bin/mvn -e exec:java
-Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
-Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building Apache Beam :: Runners :: Flink :: Examples
0.2.0-incubating-SNAPSHOT
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- exec-maven-plugin:1.4.0:java (default-cli) @
beam-runners-flink_2.10-examples ---
[WARNING]
java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Pipeline execution failed
        at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
        at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
        at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
        at
org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
        ... 6 more
Caused by: java.util.concurrent.TimeoutException: Futures timed out after
[10000 milliseconds]
        at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
        at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
        at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
        at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
        at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
        at scala.concurrent.Await$.ready(package.scala:86)
        at
org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
        at
org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
        at
org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
        at
org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
        at
org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
        at
org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
        at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
        ... 9 more
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 49.646 s
[INFO] Finished at: 2016-07-28T12:44:39+08:00
[INFO] Final Memory: 34M/451M
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
beam-runners-flink_2.10-examples: An exception occured while executing the
Java class. null: InvocationTargetException: Pipeline execution failed:
Futures timed out after [10000 milliseconds] -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute
goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on
project beam-runners-flink_2.10-examples: An exception occured while
executing the Java class. null
        at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
        at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
        at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
        at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
        at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
        at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
        at
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
        at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
        at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
        at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
        at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
        at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
        at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
        at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
        at
org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: An exception
occured while executing the Java class. null
        at
org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
        at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
        at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
        ... 20 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Pipeline execution failed
        at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
        at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
        at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
        at
org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
        ... 6 more
Caused by: java.util.concurrent.TimeoutException: Futures timed out after
[10000 milliseconds]
        at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
        at
scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
        at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
        at scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
        at
scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
        at scala.concurrent.Await$.ready(package.scala:86)
        at
org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
        at
org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
        at
org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
        at
org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
        at
org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
        at
org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
        at
org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
        ... 9 more
[ERROR]
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Best regards,
Dumi

On Wed, Jul 27, 2016 at 7:48 PM, Maximilian Michels <mxm@apache.org> wrote:

> Okay great!
>
> I think it was a permission problem before. It shouldn't matter where
> the file is as long as Flink can access it. Still puzzled about the
> error message, was that really the entire stack trace?
>
> On Wed, Jul 27, 2016 at 7:05 AM, Dumi Loghin <dumiloghin@gmail.com> wrote:
> > Hi,
> >
> > The application is running if the input file is in the examples folder:
> >
> > ~/incubator-beam/runners/flink/examples$
> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> > -Dinput=kinglear.txt -Doutput=wc-out.txt
> >
> > ( as opposed to "~/incubator-beam/runners/flink/examples$
> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> > -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt" that is throwing
> the
> > exception)
> >
> > Best regards,
> > Dumi
> >
> >
> >
> > On Tue, Jul 26, 2016 at 11:53 PM, Maximilian Michels <mxm@apache.org>
> wrote:
> >>
> >> Hi Dumi,
> >>
> >> The execution fails during shutdown of the local Flink cluster. Does
> >> the program execute before the exception occurs?
> >>
> >> Best,
> >> Max
> >>
> >> On Tue, Jul 26, 2016 at 5:15 AM, Dumi Loghin <dumiloghin@gmail.com>
> wrote:
> >> > Hi,
> >> >
> >> > Thank you!
> >> >
> >> > I have manually added WordCount.java and pom.xml to the latest Beam
> code
> >> > and
> >> > try to run it. But now I'm getting the following exception:
> >> >
> >> > :~/incubator-beam/runners/flink/examples$
> >> > /home/hadoop/apache-maven-3.3.9/bin/mvn exec:java
> >> > -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> >> > -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
> >> >
> >> > java.lang.reflect.InvocationTargetException
> >> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >> >         at
> >> >
> >> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> >> >         at
> >> >
> >> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> >> >         at java.lang.reflect.Method.invoke(Method.java:606)
> >> >         at
> >> > org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
> >> >         at java.lang.Thread.run(Thread.java:745)
> >> > Caused by: java.lang.RuntimeException: Pipeline execution failed
> >> >         at
> >> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:113)
> >> >         at
> >> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:48)
> >> >         at org.apache.beam.sdk.Pipeline.run(Pipeline.java:183)
> >> >         at
> >> >
> >> >
> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:117)
> >> >         ... 6 more
> >> > Caused by: java.util.concurrent.TimeoutException: Futures timed out
> >> > after
> >> > [10000 milliseconds]
> >> >         at
> >> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:219)
> >> >         at
> >> > scala.concurrent.impl.Promise$DefaultPromise.ready(Promise.scala:153)
> >> >         at
> >> > scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
> >> >         at
> >> > scala.concurrent.Await$$anonfun$ready$1.apply(package.scala:86)
> >> >         at
> >> >
> >> >
> scala.concurrent.BlockContext$DefaultBlockContext$.blockOn(BlockContext.scala:53)
> >> >         at scala.concurrent.Await$.ready(package.scala:86)
> >> >         at
> >> >
> >> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.shutdown(FlinkMiniCluster.scala:340)
> >> >         at
> >> >
> >> >
> org.apache.flink.runtime.minicluster.FlinkMiniCluster.stop(FlinkMiniCluster.scala:319)
> >> >         at
> >> > org.apache.flink.client.LocalExecutor.stop(LocalExecutor.java:126)
> >> >         at
> >> >
> >> >
> org.apache.flink.client.LocalExecutor.executePlan(LocalExecutor.java:194)
> >> >         at
> >> >
> >> >
> org.apache.flink.api.java.LocalEnvironment.execute(LocalEnvironment.java:90)
> >> >         at
> >> >
> >> >
> org.apache.beam.runners.flink.FlinkPipelineExecutionEnvironment.executePipeline(FlinkPipelineExecutionEnvironment.java:116)
> >> >         at
> >> > org.apache.beam.runners.flink.FlinkRunner.run(FlinkRunner.java:110)
> >> >         ... 9 more
> >> >
> >> > Best regards,
> >> > Dumi
> >> >
> >> > On Mon, Jul 25, 2016 at 7:14 PM, Maximilian Michels <mxm@apache.org>
> >> > wrote:
> >> >>
> >> >> Hi Dumi,
> >> >>
> >> >> Somewhere on the way we broke the exec configuration. I've fixed this
> >> >> in this PR and updated the README:
> >> >> https://github.com/apache/incubator-beam/pull/724
> >> >>
> >> >> Should be merged soon.
> >> >>
> >> >> Best,
> >> >> Max
> >> >>
> >> >> On Fri, Jul 22, 2016 at 2:49 PM, Lukasz Cwik <lcwik@google.com>
> wrote:
> >> >> > I think runner should be org.apache.beam.runners.flink.FlinkRunner
> >> >> > and
> >> >> > not
> >> >> > org.apache.beam.runners.flink.FlinkRunner.class but that is
> probably
> >> >> > unrelated to the failure that you are seeing.
> >> >> >
> >> >> > On Fri, Jul 22, 2016 at 7:42 AM, Dumi Loghin <dumiloghin@gmail.com
> >
> >> >> > wrote:
> >> >> >>
> >> >> >> Hi,
> >> >> >>
> >> >> >> I'm still getting an exception related to arguments. I mention
> that
> >> >> >> I'm
> >> >> >> working with commit adb472083460bc175a0685133eadb6a336f1a4a2
(22
> >> >> >> Jul).
> >> >> >>
> >> >> >> ~/incubator-beam/runners/flink/examples$ mvn exec:java
> >> >> >> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount
> >> >> >>
> >> >> >>
> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> >> >> >> --inputFile=/home/hadoop/kinglear.txt --output=wc-out.txt"
> >> >> >> ...
> >> >> >> [ERROR] Failed to execute goal
> >> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli)
on
> >> >> >> project
> >> >> >> beam-runners-flink_2.10-examples: Unable to parse configuration
of
> >> >> >> mojo
> >> >> >> org.codehaus.mojo:exec-maven-plugin:1.4.0:java for parameter
> >> >> >> arguments:
> >> >> >> Cannot store value into array: ArrayStoreException -> [Help
1]
> >> >> >> ...
> >> >> >>
> >> >> >> However, I was able to run it in early June. If I revert to,
for
> >> >> >> example,
> >> >> >> commit c2146b9f9d6a1f39a5699725ccb51829d751b88a (6 Jun) I'm
not
> >> >> >> getting
> >> >> >> the
> >> >> >> above exceptions. Maybe something changed in the options
> (arguments)
> >> >> >> handling code?
> >> >> >>
> >> >> >> Best regards,
> >> >> >> Dumi
> >> >> >>
> >> >> >> On Wed, Jul 20, 2016 at 7:35 PM, Aljoscha Krettek
> >> >> >> <aljoscha@apache.org>
> >> >> >> wrote:
> >> >> >>>
> >> >> >>> I think it should work like this:
> >> >> >>>
> >> >> >>> mvn exec:java \
> >> >> >>>
> -Dexec.mainClass=org.apache.beam.runners.flink.examples.WordCount \
> >> >> >>>
> >> >> >>>
> -Dexec.args="--runner=org.apache.beam.runners.flink.FlinkRunner.class
> >> >> >>> --inputFile='"$input"' --output='"$outfile_prefix"'"'
> >> >> >>>
> >> >> >>> On Wed, 20 Jul 2016 at 10:47 Dumi Loghin <dumiloghin@gmail.com>
> >> >> >>> wrote:
> >> >> >>>>
> >> >> >>>> Hi,
> >> >> >>>>
> >> >> >>>> I'm trying to run Wordcount example on Flink single
node but I'm
> >> >> >>>> getting
> >> >> >>>> this error:
> >> >> >>>>
> >> >> >>>> ~/incubator-beam/runners/flink/examples$ mvn exec:exec
> >> >> >>>> -Dinput=/home/hadoop/kinglear.txt -Doutput=wc-out.txt
> >> >> >>>> -Drunner=FlinkRunner
> >> >> >>>> ...
> >> >> >>>> Exception in thread "main" java.lang.IllegalArgumentException:
> No
> >> >> >>>> Runner
> >> >> >>>> was specified and the DirectRunner was not found on
the
> classpath.
> >> >> >>>> Specify a runner by either:
> >> >> >>>>     Explicitly specifying a runner by providing the
'runner'
> >> >> >>>> property
> >> >> >>>>     Adding the DirectRunner to the classpath
> >> >> >>>>     Calling 'PipelineOptions.setRunner(PipelineRunner)'
directly
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:282)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.PipelineOptions$DirectRunner.create(PipelineOptions.java:273)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.ProxyInvocationHandler.getDefault(ProxyInvocationHandler.java:482)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:157)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.PipelineOptionsValidator.validate(PipelineOptionsValidator.java:72)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:306)
> >> >> >>>> at
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> org.apache.beam.runners.flink.examples.WordCount.main(WordCount.java:106)
> >> >> >>>> ...
> >> >> >>>>
> >> >> >>>>
> >> >> >>>> I've tried with -Drunner=FlinkRunner and without it
and I get
> the
> >> >> >>>> same
> >> >> >>>> error. Moreover, the runner is also set in the source:
> >> >> >>>>
> >> >> >>>> options.setRunner(FlinkRunner.class);
> >> >> >>>>
> >> >> >>>> Any suggestions?
> >> >> >>>>
> >> >> >>>> Best regards,
> >> >> >>>> Dumi
> >> >> >>
> >> >> >>
> >> >> >
> >> >
> >> >
> >
> >
>

Mime
View raw message