storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eugene <edvor...@gmail.com>
Subject Re: Unit testing topologies
Date Sat, 05 Jul 2014 03:11:19 GMT
I did session at storm meetup in NYC about testing storm topologies.
I personally use Groovy and Spock  for
all my testing. Groovy is easy to pick up for any java programmer and
provides big productivity boost in writing tests..
Spock practically replacing  junit, mockito and have BDD style of writing
tests.
Slides for my presentation available here
<http://www.slideshare.net/edvorkin/testing-storm-components-with-groovy-and-spock#>

.
Basic idea is that you can easily mock tuples and test each bolt in
isolation. You can also replace outputCollector with your implementation so
you test what is your bolt emitting. To test topology end-to-end and
interaction between components you have to write integration test. Trigger
event and check what you get at the end.
Storm Real-time Processing Cookbook has couple chapters about testing.
Hope that help
Eugene Dvorkin.
NYC Storm UG Organizer


On Fri, Jul 4, 2014 at 12:24 PM, Andrew Montalenti <andrew@parsely.com>
wrote:

> It's somewhat old but there is a great series of blog posts about testing
> Storm topologies using Clojure:
>
> http://www.pixelmachine.org/2011/12/17/Testing-Storm-Topologies.html
>
> and if you take a look at Storm's testing.clj file, there are some great
> Clojure utilities used for testing within Storm itself:
>
>
> https://github.com/apache/incubator-storm/blob/master/storm-core/src/clj/backtype/storm/testing.clj
>
> I learned a bunch by reading the multilang test cases here as a starting
> point:
>
>
> https://github.com/apache/incubator-storm/blob/master/storm-core/test/clj/backtype/storm/multilang_test.clj
>
> and then learning more about the Clojure DSL:
>
> http://storm.incubator.apache.org/documentation/Clojure-DSL.html
>
> Of course, if you're a Java programmer, you may be looking at all this
> Clojure-based testing infrastructure and saying, "Great, now I have two
> problems -- my unit tests, and learning Clojure." Indeed :) But, Clojure is
> a pretty nice language and worth learning for use cases like this.
>
> ---
> Andrew Montalenti
> Co-Founder & CTO
> http://parse.ly
>
>
> On Thu, Jul 3, 2014 at 3:41 AM, Richards Peter <hbkrichards@gmail.com>
> wrote:
>
>> Hi,
>>
>> Could anyone suggest me some more good materials for unit testing storm
>> topologies?
>>
>> I found the following code:
>>
>> https://github.com/xumingming/storm-lib/blob/master/src/jvm/storm/TestingApiDemo.java
>>
>> I would be nice if anyone can present a talk about unit testing
>> topologies in any meetup/event. Will it be possible for anyone to conduct
>> such sessions?
>>
>> Thanks,
>> Richards Peter.
>>
>
>


-- 
Eugene Dvorkin
Software Engineer
New York City Storm User Group - organizer
WebMD
email: edvorkin@gmail.com
phone: 646-250-9649
eugenedvorkin.com
Connect with me on:
LinkedIn <http://www.linkedin.com/in/eugenedvorkin> Twitter
<http://twitter.com/edvorkin>

Mime
View raw message