streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suneel Marthi <suneel.mar...@gmail.com>
Subject Re: Update to continuous integration flow
Date Fri, 21 Oct 2016 04:47:42 GMT


Sent from my iPhone

> On Oct 20, 2016, at 11:34 PM, sblackmon <sblackmon@apache.org> wrote:
> 
> Jenkins jobs streams-project-verify and streams-examples-verify are passing as well now,
including all of the new live data collection, docker database, and flink example integration
tests.

U might want to reach out to the flink folks to add streams project to their 'Powered by Flink'
page.
> 
> streams-project-verify is a bit particular only because the facebook credentials it’s
been given have a very quick expiration.  So until we source more stable credentials to use
for testing, a new token will need to be provider prior to any execution where we’re looking
for 100% integration test pass rate - probably just builds where something has changed in
that module or built from a release candidate.
> 
> streams-project-site and streams-examples-site have been configured to run on SCM updates,
but at the moment they are not setup to auto-deploy to the website - that’s still a documented
manual process.  You can however click to view the staging site in jenkins.
> 
> Steve
> On October 18, 2016 at 10:58:07 PM, sblackmon (sblackmon@apache.org) wrote:
> 
> Update:
> 
> Jenkins jobs streams-master streams-project and streams-examples are blue with no test
failures.
> 
> Jenkins jobs streams-project-verify and streams-examples-verify have been created and
configured.
> 
> I’ve observed them successfully launching and reading/writing to docker containers,
as well as collecting data from twitter using credentials supplied by the job.
> 
> After merging the open PRs and a bit more tinkering in Jenkins, I expect those jobs to
be blue as well with all ITs passing project-wide.
> 
> Steve
> 
> On October 17, 2016 at 3:42:04 PM, sblackmon (sblackmon@apache.org) wrote:
> 
> Hello,
> 
> I’m working on the streams jenkins setup to increase our CI coverage and support more
types of testing.
> 
> What’s changed code wise to precipitate changes?
> 
> - streams-master and streams-examples are now release artifacts and need CI
> - some providers now contain integration tests that collect data live, but require working
api credentials to do so
> - some persisters (and all examples) now contain integration tests that launch supporting
services with docker
> - streams-project now contains modules that build maven plugins, and has testing that
relies on those plugins
> 
> What’s changing? 
> 
> - integration testing (mvn verify) is being isolated from standard build (mvn install)
> - integration testing steps will be configured to launch docker containers and use test
credentials to run the live data tests
> - snapshots and site snapshots will publish as separate steps, contingent on stability
of standard builds
> 
> Attached a diagram of the new build flow, which will be including on the continuous integration
page of the website.
> 
> Dashed line: checkout
> Dotted line: jenkins upstream/downstream relationships
> Solid line: deployment of artifacts
> 
> Hint: you can click each node in the SVG to jump to a relevant URL.
> 
> I’m working on setting this up and hope to have everything fully working before the
0.4 release.  In the meantime you may see build failures as jobs and test plans are being
refactored in STREAMS-429 branches.
> 
> Steve
> 
> 

Mime
View raw message