storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Xor <andreas.gramme...@gmail.com>
Subject Re: Found multiple defaults.yaml resources
Date Wed, 18 Feb 2015 06:11:33 GMT
I concur to this; as a rule of the thumb I use compile when I develop
locally (in LocalCluster) and provided if I deploy the topology to an
actual cluster.

Regards.

On Wed, Feb 18, 2015 at 7:23 AM, José Barrueta <jose@stormpath.com> wrote:

> Hey Nikon, I had the same issue days ago, the problem in my case was that
> I wasn't declaring the storm-core dependency as provided in the pom file,
> therefore the fat jar included the dependency, which is provided by nimbus
> (and workers) when deploying in the cluster.
>
> ...
>         <dependency>
>             <groupId>org.apache.storm</groupId>
>             <artifactId>storm-core</artifactId>
>             <version>${storm.version}</version>
>             <scope>provided</scope>
>         </dependency>
> ...
>
> Best,
>
> Jose Luis
>
> On Tue, Feb 17, 2015 at 5:29 PM, Nick R. Katsipoulakis <
> nick.katsip@gmail.com> wrote:
>
>> Hello,
>>
>> I am currently trying to automate some experiments on Storm and to do so,
>> I have created a Java process which in turn launches three Java
>> sub-processes using the ProcessBuilder framework. The three processes are
>> mainly executions of jar files (2 of them simple java applications, 1 of
>> them Storm topology submitted to a Storm cluster). Therefore, the 2 simple
>> java applications are executed as terminal commands:
>>
>> $>java -jar executable-jar <arg-1> <arg-2> ... <arg-N>
>>
>> The storm topology is bundled into a jar using Eclipse maven package
>> assembly:single goal and the following plugin configuration is added to the
>> pom.xml of the project:
>>
>> <build>
>>
>> <plugins>
>>
>> <plugin>
>>
>> <groupId>org.apache.maven.plugins</groupId>
>>
>> <artifactId>maven-assembly-plugin</artifactId>
>>
>> <version>2.5.3</version>
>>
>> <configuration>
>>
>> <descriptorRefs>
>>
>> <descriptorRef>jar-with-dependencies</descriptorRef>
>>
>> </descriptorRefs>
>>
>> <archive>
>>
>> <manifest>
>>
>> <mainClass>gr.katsip.storm.topology.ExperimentalTopology</mainClass>
>>
>> </manifest>
>>
>> </archive>
>>
>> </configuration>
>>
>> <executions>
>>
>> <execution>
>>
>> <phase>package</phase>
>>
>> <goals>
>>
>> <goal>single</goal>
>>
>> </goals>
>>
>> </execution>
>>
>> </executions>
>>
>>         </plugin>
>>
>> </plugins>
>>
>> </build>
>>
>>
>> The problem is that when I create the Java sub-process that submits the
>> following command:
>>
>>
>> $>storm jar bundled-topology.jar <arg-1> <arg-2> ... <arg-M>
>>
>>
>> at some point I get the following error:
>>
>>
>> Exception in thread "main" java.lang.RuntimeException: Found multiple
>> defaults.yaml resources. You're probably bundling the Storm jars with your
>> topology jar.
>> [jar:file:/Users/nick/Documents/stream-gen/Experiment/synefo-topology.jar!/defaults.yaml,
>> jar:file:/Users/nick/apache-storm/lib/storm-core-0.9.2-incubating.jar!/defaults.yaml]
>>
>> at backtype.storm.utils.Utils.findAndReadConfigFile(Utils.java:140)
>>
>> at backtype.storm.utils.Utils.readDefaultConfig(Utils.java:167)
>>
>> at backtype.storm.utils.Utils.readStormConfig(Utils.java:191)
>>
>> at backtype.storm.config$read_storm_config.invoke(config.clj:121)
>>
>> at backtype.storm.command.config_value$_main.invoke(config_value.clj:22)
>>
>> at clojure.lang.AFn.applyToHelper(AFn.java:161)
>>
>> at clojure.lang.AFn.applyTo(AFn.java:151)
>>
>>  at backtype.storm.command.config_value.main(Unknown Source)
>>
>> Therefore, I understand that during packaging, maven includes a
>> default.yaml file in my package, that conflicts with the yam file of the
>> cluster.
>>
>> My question is how can I remove the default.yaml from my bundled-topology
>> or how can I overcome this issue. The simple solution would be to start my
>> topology by itself and then run the other two processes, but I want my
>> experiments to be fully automated.
>>
>> Thank you and I apologize in advance for the long post.
>>
>> Cheers,
>> Nikos
>>
>> --
>> Nikolaos Romanos Katsipoulakis,
>> University of Pittsburgh, PhD candidate
>>
>
>

Mime
View raw message