flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax" <mj...@informatik.hu-berlin.de>
Subject Re: Some new problems about flink-storm-compatibility
Date Thu, 20 Aug 2015 11:47:49 GMT
Hi Huang,

you are using the wrong jar file. The current built, does not assemble a
jar file for StormWordCountLocal. You can extend pom.xml. Look at the
example StormWordCountRemoteBySubmitter and the corresponding assembly
file word-count-storm.xml.

What you need to do is:
  - add a new <plugin> tag similar to StormWordCountRemoteBySubmitter
   -> change descriptor file, program-class, and id
 - copy  word-count-storm.xml to new descriptor file and adjust it
accordingly (needs to include StormWordCountLocal instead of
StormWordCountRemoteBySubmitter (and maybe other adjustments)
 - run maven -DskipTests package to build the new jar file

If you have further problems, just let us know.

-Matthias


On 08/20/2015 01:19 PM, huangwei (G) wrote:
> Hi,
> 
> I got some new problems about the storm compatibility currently.
> These occurred when I ran the “storm-wordcount” in the storm compatibility on a flink-0.10SNAPSHOT
which I built it over a latest flink project.
> 
> 
> First, I start a local flink:
> 
> $ cd bin
> 
> $ ./start-local.sh
> 
> 
> 
> Then I ran the example:
> 
> $ bin/flink run --jarfile ./examples/flink-storm-compatibility-examples-0.10-SNAPSHOT.jar
-c org.apache.flink.stormcompatibility.wordcount.StormWordCountLocal
> 
> 
> 
> It took a trouble like following:
> 
> Executing StormWordCount* example with built-in default data
> 
>   Provide parameters to read input data from a file
> 
>   Usage: StormWordCount* <text path> <result path>
> 
> java.lang.ExceptionInInitializerError
> 
> at backtype.storm.topology.TopologyBuilder.createTopology(TopologyBuilder.java:106)
> 
> at org.apache.flink.stormcompatibility.api.FlinkTopologyBuilder.createTopology(FlinkTopologyBuilder.java:78)
> 
> at org.apache.flink.stormcompatibility.wordcount.StormWordCountLocal.main(StormWordCountLocal.java:66)
> 
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 
> at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> 
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 
> at java.lang.reflect.Method.invoke(Method.java:497)
> 
> at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:437)
> 
> at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:353)
> 
> at org.apache.flink.client.program.Client.run(Client.java:327)
> 
> at org.apache.flink.client.CliFrontend.executeProgram(CliFrontend.java:608)
> 
> at org.apache.flink.client.CliFrontend.run(CliFrontend.java:296)
> 
> at org.apache.flink.client.CliFrontend.parseParameters(CliFrontend.java:927)
> 
> at org.apache.flink.client.CliFrontend.main(CliFrontend.java:977)
> 
> Caused by: java.lang.RuntimeException: Could not find config file on classpath defaults.yaml
> 
> at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:129)
> 
> at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:160)
> 
> at backtype.storm.utils.Utils.readStormConfig(Utils.java:184)
> 
> at backtype.storm.utils.Utils.<clinit>(Utils.java:71)
> 
> ... 14 more
> 
> 
> 
> And then I added the default.yaml into flink-storm-compatibility-examples-0.10-SNAPSHOT.jar.
> 
> It took another issue in the client log:
> 
> 18:55:42,526 INFO  org.apache.flink.runtime.jobmanager.JobManager                - Status
of job cb8ec2988613290d31eb6496c0af2e29 (Streaming WordCount) changed to FAILED.
> 
> java.lang.Exception: Call to registerInputOutput() of invokable failed
> 
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:526)
> 
> at java.lang.Thread.run(Thread.java:745)
> 
> Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Cannot load
user class: org.apache.flink.stormcompatibility.wrappers.StormBoltWrapper
> 
> ClassLoader info: URL ClassLoader:
> 
> Class not resolvable through given classloader.
> 
> at org.apache.flink.streaming.api.graph.StreamConfig.getStreamOperator(StreamConfig.java:219)
> 
> at org.apache.flink.streaming.runtime.tasks.StreamTask.registerInputOutput(StreamTask.java:135)
> 
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:523)
> 
> ... 1 more
> 
> 
> 
> I have no idea to fix this and need some help.
> 
> 
> 
> Thanks!
> 
> 
> 
> Greetings,
> Huang Wei
> 华为技术有限公司 Huawei Technologies Co., Ltd.
> 
> 
> Tel:+86 18106512602
> Email:huangwei111@huawei.com
> 


Mime
View raw message