streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Douglas <robert.baker.doug...@gmail.com>
Subject Re: [DISCUSS] Official library of examples
Date Thu, 12 Feb 2015 16:08:23 GMT
Hey Steve, I think this is a great idea. Streams has a ton of use cases and
I think to a new user, they can be a bit daunting. I'd be happy to help
shore up some of the examples.

-- Robert

On Wed, Feb 11, 2015 at 9:53 AM, Steve Blackmon <sblackmon@apache.org>
wrote:

> Reviving an old discussion:
>
> I propose that we create an official streams-examples repository in the
> apache git repository with the following characteristics:
>
>    - maven multi-level multi-module project
>    - reference streams-project as parent
>    - first level : runtime (local, dropwizard, pig, storm, etc...)
>    - below, individual examples modules built upon that runtime.
>
> Each example would:
>
>    - Contain a primary class which uses a StreamsBuilder to launch an
>    example stream, or set of complementary streams.
>    - Alternatively, contain a script suitable for execution within a
>    specific runtime (pig/spark shell for example)
>    - Optionally, contain processors and utility classes suited to a
>    specific purpose within
>    - Optionally, override the behavior of providers, processors, utility
>    classes from the core project to achieve a specific purpose
>    - Contain a configuration class which declares all required and optional
>    configuration
>    - Contain a README.md conforming to a template TBD, describing what the
>    example does
>    - Contain at least one example configuration in src/main/resources and
>    in README.md
>    - Contain a machine readable graph definition (dot or suitable
>    alternative) and png version visible in README.md
>    - Contain integration test(s) demonstrating that a small input set
>    traverses the stream, resulting in expected outputs and (optionally)
>    properly handled exceptions.
>    - Describe requirements for a successful deployment and execution in
>    README.md.
>    - Build an executable uber-jar
>    - Build an executable docker image
>
> https://github.com/steveblackmon/streams-examples is a collection of
> modules that conform in varying degrees to this pattern.  Less organized
> and less tested than I'm proposing, but containing code & poms that are
> minimal, readable, and useful "out-of-the-box".  I'll commit to increasing
> the quality of 5-10 of them and submitting them to
>
> Anyone agree this is a good idea? Have concerns? Want to help?
>
> Steve Blackmon
> sblackmon@apache.org
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message