streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Blackmon <sblack...@apache.org>
Subject Re: DISCUSS: Consolidate Example modules with Project Repo
Date Tue, 16 Jan 2018 18:12:49 GMT
On Jan 16, 2018 at 3:57 AM, Ate Douma <ate@douma.nu> wrote:



On 2018-01-16 05:58, Steve Blackmon wrote:

Has anyone had a chance to take a look?

I’m content with this change-set, and I am planning to lazy consensus merge
in 48 hours.

Following that I’ll ask infra to delete the legacy streams-examples repo in
git asf gitbox and github.


If I understand this correctly, you just *copied* the current examples
files, right?
That means that all commit history in the streams-example repo, once
deleted, will be gone as well.
Not sure I or anyone cares, just want to make sure this is the intent?

Ate


That’s correct.  This will effectively squash all the commits in
streams-examples into a single commit in a sub-module of apache-streams.

I suppose there’s no pressing need to delete the streams-examples repo
right away, other than to avoid user confusion, which could be mitigated
with a deprecation notice and at the top of the root README.

Steve



Steve

On Jan 10, 2018 at 12:16 AM, Steve Blackmon <sblackmon@apache.org> wrote:


Having received three +1 and no objections, I took the liberty of creating
STREAMS-574 ticket and opening a pull request.

https://issues.apache.org/jira/browse/STREAMS-574
https://github.com/apache/streams/pull/417

It’s a straight lift-and-shift from apache/streams-examples into a
sub-module of apache/streams.

The only changes made in transit were trivial, not worth enumerating, and
necessary to get the ITs passing.

One module - twitter-follow-neo4j - looks to some additional work to get
the ITs passing, so I’ve removed it from the reactor (temporarily) in this
PR.

Please take a look at your leisure.

Steve

On Jan 9, 2018 at 11:12 AM, Steve Blackmon <sblackmon@apache.org> wrote:


I think we would compile examples with every pull request and merge to
master, and run examples integration tests along with the rest of the ITs
in the streams-project-verify jenkins job (and as a step in the release
process like you mentioned.)

We’re now at the point where we need to start categorizing our integration
tests more finely and giving the developer more control over which run -
this is because some integration tests require credentials with special
permissions that not every developer will have access to. I think we can
fit the examples into that scheme however is appropriate, while also making
their execution conditional on a separate profile disabled by default.

Steve

Sent from Astro <https://www.helloastro.com> for Mac


On Jan 9, 2018 at 10:24 AM, Joey Frazee <joey.frazee@icloud.com> wrote:


+1

Is the plan to have them be part of every build or to use an optional build
profile? I’m partial to the latter; that said, I could see the argument
wanting whatever the clean install is to be how we build and test for an
Apache release since we will now need those to be voted on for every
release.

-joey

On Jan 8, 2018, 1:44 PM -0700, Trevor Grant <trevor.d.grant@gmail.com>,
wrote:

+1

On Mon, Jan 8, 2018 at 2:27 PM, Suneel Marthi <smarthi@apache.org> wrote:

+1

On Mon, Jan 8, 2018 at 3:26 PM, Steve Blackmon <sblackmon@apache.org
wrote:

I’d like to discuss consolidating apache/streams-examples into
apache/streams under a new top-level module ‘examples’.

We had some good reasons for keeping them isolated during incubation, but
I’ve come around to the perspective that the significant simplification

to

our setup, test, CI, website publish, and release processes we would get

by

merging them in would outweigh all of the benefits of keeping them apart.

Also I think the negatives of having the examples reside inside the

project

can largely be mitigated by adding a new profile that shades and
integration tests the examples but is disabled by default.

Lastly, given that Flink, Kafka, and Spark are all getting by with their
examples hosted inside their primary SCM, I see no reason Streams

couldn’t

do the same

Any +1 or alternative perspectives on the matter?

Steve

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message