streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Blackmon <sblack...@apache.org>
Subject [DISCUSS] Official library of examples
Date Wed, 11 Feb 2015 15:53:13 GMT
Reviving an old discussion:

I propose that we create an official streams-examples repository in the
apache git repository with the following characteristics:

   - maven multi-level multi-module project
   - reference streams-project as parent
   - first level : runtime (local, dropwizard, pig, storm, etc...)
   - below, individual examples modules built upon that runtime.

Each example would:

   - Contain a primary class which uses a StreamsBuilder to launch an
   example stream, or set of complementary streams.
   - Alternatively, contain a script suitable for execution within a
   specific runtime (pig/spark shell for example)
   - Optionally, contain processors and utility classes suited to a
   specific purpose within
   - Optionally, override the behavior of providers, processors, utility
   classes from the core project to achieve a specific purpose
   - Contain a configuration class which declares all required and optional
   configuration
   - Contain a README.md conforming to a template TBD, describing what the
   example does
   - Contain at least one example configuration in src/main/resources and
   in README.md
   - Contain a machine readable graph definition (dot or suitable
   alternative) and png version visible in README.md
   - Contain integration test(s) demonstrating that a small input set
   traverses the stream, resulting in expected outputs and (optionally)
   properly handled exceptions.
   - Describe requirements for a successful deployment and execution in
   README.md.
   - Build an executable uber-jar
   - Build an executable docker image

https://github.com/steveblackmon/streams-examples is a collection of
modules that conform in varying degrees to this pattern.  Less organized
and less tested than I'm proposing, but containing code & poms that are
minimal, readable, and useful "out-of-the-box".  I'll commit to increasing
the quality of 5-10 of them and submitting them to

Anyone agree this is a good idea? Have concerns? Want to help?

Steve Blackmon
sblackmon@apache.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message