samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Williams <william...@gmail.com>
Subject Re: improving hello-samza / testing
Date Wed, 17 Jun 2015 10:59:34 GMT
Thanks for the great feedback!  the SAMZA-681 patch looks great, when
it's merged it'd be nice to add some tests to hello-samza that
demonstrate it...

Thanks again,
--tim

On Wed, Jun 17, 2015 at 12:17 AM, Chris Riccomini <criccomini@apache.org> wrote:
> Hey Tim,
>
> This is a really good discussion to have. The testing that I've seen with
> Samza falls into two categories:
>
> 1. Instantiate your StreamTask, and mock all params in the process()/init()
> methods.
> 2. A mini-ontegration test that starts ZooKeeper, and Kafka, and feeds
> messages into a topic, and validates it gets messages back out from the
> output topic.
> 3. A full blown integration test that uses Zopkio.
>
> For an example of (2), in practice, have a look at TestStatefuleTask:
>
>
> https://git-wip-us.apache.org/repos/asf?p=samza.git;a=blob;f=samza-test/src/test/scala/org/apache/samza/test/integration/TestStatefulTask.scala;h=ea702a919348305ff95ce0b4ca1996a13aff04ec;hb=HEAD
>
> As you can see, writing this kind of integration test can be a bit painful.
>
> (3) is documented here:
>
>   http://samza.apache.org/contribute/tests.html
>
> Another way to test would be to start a full-blown container using
> ThreadJobFactory/ProcessJobFactory, but use a MockSystemFactory to mock out
> the system consumer/system producer.
>
> Has anyone else tested Samza in other ways?
>
> Cheers,
> Chris
>
> On Tue, Jun 16, 2015 at 11:00 AM, Tim Williams <williamstw@gmail.com> wrote:
>
>> I'm learning samza by the hello-samza project and notice the lack of
>> tests.  Where's a good place to learn how folks are properly testing
>> things written with samza?
>>
>> Thanks,
>> --tim
>>

Mime
View raw message