beam-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Amit Sela <amitsel...@gmail.com>
Subject Re: SparkPipelineRunner ERROR
Date Wed, 25 May 2016 08:00:18 GMT
I'll take a look first chance I get and get back to you.

On Wed, May 25, 2016, 10:57 Punit Naik <naik.punit44@gmail.com> wrote:

> Hi Amit
>
> Thanks, it ran but got stuck at:
>
> [INFO] --- exec-maven-plugin:1.4.0:exec (default-cli) @ spark ---
> log4j:WARN No appenders could be found for logger
> (org.apache.beam.sdk.options.PipelineOptionsFactory).
> log4j:WARN Please initialize the log4j system properly.
> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for
> more info.
>
>
> On Wed, May 25, 2016 at 1:17 PM, Amit Sela <amitsela33@gmail.com> wrote:
>
>> I'm not near a computer but are you running from the parent project dir
>> or the runner ? You need to run from the runner dir.
>>
>> On Wed, May 25, 2016, 10:44 Punit Naik <naik.punit44@gmail.com> wrote:
>>
>>> Hi Amit
>>>
>>> Just tried that and still got an error:
>>>
>>> [ERROR] Failed to execute goal
>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec (default-cli) on project
>>> parent: The parameters 'executable' for goal
>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:exec are missing or invalid ->
>>> [Help 1]
>>>
>>> On Wed, May 25, 2016 at 1:01 PM, Amit Sela <amitsela33@gmail.com> wrote:
>>>
>>>> Would you try running with mvn exec:exec ?
>>>> See here:
>>>> https://github.com/apache/incubator-beam/blob/master/runners/spark/README.md
>>>>
>>>> On Wed, May 25, 2016, 10:14 Punit Naik <naik.punit44@gmail.com> wrote:
>>>>
>>>>> Hi Everyone
>>>>>
>>>>> I compiled beam project with the command "mvn clean install
>>>>> -DskipTests" and then ran the WordCount example with the following command:
>>>>>
>>>>> mvn compile exec:java -pl examples/java
>>>>> -Dexec.mainClass=org.apache.beam.examples.WordCount
>>>>> -Dexec.args="--runner=SparkPipelineRunner --sparkMaster=local
>>>>> --inputFile=/in --output=/beam-out"
>>>>>
>>>>> But it gave me an error saying:
>>>>>
>>>>> [ERROR] Failed to execute goal
>>>>> org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli) on project
>>>>> java-examples-all: An exception occured while executing the Java class.
>>>>> null: InvocationTargetException: Unknown 'runner' specified
>>>>> 'SparkPipelineRunner', supported pipeline runners
>>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>>> DirectPipelineRunner] -> [Help 1]
>>>>> org.apache.maven.lifecycle.LifecycleExecutionException: Failed to
>>>>> execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (default-cli)
>>>>> on project java-examples-all: An exception occured while executing the
Java
>>>>> class. null
>>>>>     at
>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:212)
>>>>>     at
>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
>>>>>     at
>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
>>>>>     at
>>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
>>>>>     at
>>>>> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
>>>>>     at
>>>>> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
>>>>>     at
>>>>> org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
>>>>>     at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
>>>>>     at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
>>>>>     at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
>>>>>     at org.apache.maven.cli.MavenCli.execute(MavenCli.java:863)
>>>>>     at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:288)
>>>>>     at org.apache.maven.cli.MavenCli.main(MavenCli.java:199)
>>>>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>     at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>     at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>     at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>     at
>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
>>>>>     at
>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
>>>>>     at
>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
>>>>>     at
>>>>> org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
>>>>> Caused by: org.apache.maven.plugin.MojoExecutionException: An
>>>>> exception occured while executing the Java class. null
>>>>>     at
>>>>> org.codehaus.mojo.exec.ExecJavaMojo.execute(ExecJavaMojo.java:345)
>>>>>     at
>>>>> org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
>>>>>     at
>>>>> org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:207)
>>>>>     ... 20 more
>>>>> Caused by: java.lang.reflect.InvocationTargetException
>>>>>     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>>     at
>>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>>>>>     at
>>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>>>>>     at java.lang.reflect.Method.invoke(Method.java:498)
>>>>>     at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:293)
>>>>>     at java.lang.Thread.run(Thread.java:745)
>>>>> Caused by: java.lang.IllegalArgumentException: Unknown 'runner'
>>>>> specified 'SparkPipelineRunner', supported pipeline runners
>>>>> [BlockingDataflowPipelineRunner, DataflowPipelineRunner,
>>>>> DirectPipelineRunner]
>>>>>     at
>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1434)
>>>>>     at
>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.access$400(PipelineOptionsFactory.java:107)
>>>>>     at
>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory$Builder.as(PipelineOptionsFactory.java:292)
>>>>>     at org.apache.beam.examples.WordCount.main(WordCount.java:192)
>>>>>     ... 6 more
>>>>> Caused by: java.lang.ClassNotFoundException: SparkPipelineRunner
>>>>>     at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
>>>>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>>>>>     at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>>>>>     at java.lang.Class.forName0(Native Method)
>>>>>     at java.lang.Class.forName(Class.java:264)
>>>>>     at
>>>>> org.apache.beam.sdk.options.PipelineOptionsFactory.parseObjects(PipelineOptionsFactory.java:1421)
>>>>>     ... 9 more
>>>>>
>>>>> What is the problem?
>>>>>
>>>>>
>>>>> --
>>>>> Thank You
>>>>>
>>>>> Regards
>>>>>
>>>>> Punit Naik
>>>>>
>>>>
>>>
>>>
>>> --
>>> Thank You
>>>
>>> Regards
>>>
>>> Punit Naik
>>>
>>
>
>
> --
> Thank You
>
> Regards
>
> Punit Naik
>

Mime
View raw message