streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Blackmon <st...@blackmon.org>
Subject Re: [VOTE] Official library of examples
Date Fri, 13 Feb 2015 17:57:10 GMT
+1

Regarding location of the examples, personally I think it makes more sense
to put them in a separate repo. Main reasons are:

   1. Accessibility.  Users who just want to learn about how to compose
   streams or run pre-built streams should be able to just check out and run
   the examples without downloading streams.
   2. Build time.  Adding tens of examples with tests and shading to the
   core project would add minutes to an already painfully long build.
   3. Binaries.  Examples should build executable binaries, and it's
   preferable i think to isolate the core project from licensing requirements
   that impact binaries.
   4. Testing.  If a change in core project breaks one or more examples,
   refactoring all of them shouldn't be a blocker to merging that PR.
   However, refactoring until all examples have passing tests must occur
   before the next release of the examples (where the streams-project version
   would increment to the latest release of streams-project)

Steve

On Fri, Feb 13, 2015 at 10:43 AM, Daniel Sullivan <dbsullivan23@gmail.com>
wrote:

> +1
>
> Am I correct in understanding that the examples will be under the the
> incubator-streams repo somewhere?
>
> On Fri, Feb 13, 2015 at 4:48 PM, Steve Blackmon <sblackmon@apache.org>
> wrote:
>
> > Please vote in favor of provisioning a new empty git repo named
> > streams-examples associated with this project.
> >
> > The discussion below will inform the initial commit, of a basic pom
> > hierarchy and README documenting the guidelines for adding examples.
> >
> > Examples that meet community standards will be welcome from all Apache
> > contributors.
> >
> > Vote open for 72 hours.
> >
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
> > Steve Blackmon
> > sblackmon@apache.org
> >
> > On Thu, Feb 12, 2015 at 10:11 AM, Matt Franklin <
> m.ben.franklin@gmail.com>
> > wrote:
> >
> > > On Thu Feb 12 2015 at 11:09:35 AM Robert Douglas <
> > > robert.baker.douglas@gmail.com> wrote:
> > >
> > > > Hey Steve, I think this is a great idea. Streams has a ton of use
> cases
> > > and
> > > > I think to a new user, they can be a bit daunting. I'd be happy to
> help
> > > > shore up some of the examples.
> > > >
> > >
> > > I would recommend having infra setup a separate git repository and
> adding
> > > modules to that from scratch, rather than importing them from another
> > > repository.
> > >
> > >
> > > >
> > > > -- Robert
> > > >
> > > > On Wed, Feb 11, 2015 at 9:53 AM, Steve Blackmon <
> sblackmon@apache.org>
> > > > wrote:
> > > >
> > > > > Reviving an old discussion:
> > > > >
> > > > > I propose that we create an official streams-examples repository
in
> > the
> > > > > apache git repository with the following characteristics:
> > > > >
> > > > >    - maven multi-level multi-module project
> > > > >    - reference streams-project as parent
> > > > >    - first level : runtime (local, dropwizard, pig, storm, etc...)
> > > > >    - below, individual examples modules built upon that runtime.
> > > > >
> > > > > Each example would:
> > > > >
> > > > >    - Contain a primary class which uses a StreamsBuilder to launch
> an
> > > > >    example stream, or set of complementary streams.
> > > > >    - Alternatively, contain a script suitable for execution within
> a
> > > > >    specific runtime (pig/spark shell for example)
> > > > >    - Optionally, contain processors and utility classes suited to
a
> > > > >    specific purpose within
> > > > >    - Optionally, override the behavior of providers, processors,
> > > utility
> > > > >    classes from the core project to achieve a specific purpose
> > > > >    - Contain a configuration class which declares all required and
> > > > optional
> > > > >    configuration
> > > > >    - Contain a README.md conforming to a template TBD, describing
> > what
> > > > the
> > > > >    example does
> > > > >    - Contain at least one example configuration in
> src/main/resources
> > > and
> > > > >    in README.md
> > > > >    - Contain a machine readable graph definition (dot or suitable
> > > > >    alternative) and png version visible in README.md
> > > > >    - Contain integration test(s) demonstrating that a small input
> set
> > > > >    traverses the stream, resulting in expected outputs and
> > (optionally)
> > > > >    properly handled exceptions.
> > > > >    - Describe requirements for a successful deployment and
> execution
> > in
> > > > >    README.md.
> > > > >    - Build an executable uber-jar
> > > > >    - Build an executable docker image
> > > > >
> > > > > https://github.com/steveblackmon/streams-examples is a collection
> of
> > > > > modules that conform in varying degrees to this pattern.  Less
> > > organized
> > > > > and less tested than I'm proposing, but containing code & poms
that
> > are
> > > > > minimal, readable, and useful "out-of-the-box".  I'll commit to
> > > > increasing
> > > > > the quality of 5-10 of them and submitting them to
> > > > >
> > > > > Anyone agree this is a good idea? Have concerns? Want to help?
> > > > >
> > > > > Steve Blackmon
> > > > > sblackmon@apache.org
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message