couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ilya Khlopotov" <iil...@ca.ibm.com>
Subject Re: [PROPOSAL] Improving the quality of CouchDB
Date Thu, 10 Sep 2015 13:52:44 GMT

One more option to consider (based on Russell idea expressed personally).
We could follow following process when we create PR.
1. create PR against dependency (or multiple deps)
2. create PR (this is not for merge) for couchdb repo which updates
rebar.config.script to point to PR created in #1
3. travis would be pointed to couchdb repo only and will build and test
whole project.

This approach would simplify the review job as well since he/she don't need
to manually checkout right versions of updated dependencies.

We can go further with this approach by converting couchdb repository into
meta repository which would only have meta information for release.
Here is what I'm talking about https://github.com/iilyak/couchdb-manifest

BR,
ILYA



From:	Dale Harvey <dale@arandomurl.com>
To:	dev@couchdb.apache.org
Date:	2015/09/10 02:40 AM
Subject:	Re: [PROPOSAL] Improving the quality of CouchDB



I dont think CI is a dream, It should take an afternoon at most to get
CouchDB setup on travis on a single platform to ensure no major regressions
come though. If anyone wants help doing that feel free to ping me on
#pouchdb / #couchdb, we already test couchdb master in travis.

I do think multirepos is an issue and that solution is not so simple, we
went through the same issues with pouchdb as we attempted to split out our
repository.

The basic problem is, X is a downstream component of CouchDB in its own
repo, X may have its own unit tests etc, however I can still make a change
to X, have it all passing but it can still complete break the CouchDB
suite. This also entirely break git bisect so not only are breakages more
likely but they are significantly harder to debug.

The 2 main solutions to that work to some degree for us

1. Dont split out into lots of repositories, if you put those components
inside the CouchDB repo, then they will get the CouchDB tests run against
them when changes are made and you wont break the CouchDB repo.

2. Anything that does live outside the CouchDB repo, pin their version
inside the CouchDB repo, dont have commits to subproject X automatically be
applied to CouchDB, that means you can commit whatever you want to X but
CouchDB will still be working, when you come to update the version of X you
pinned, you can see that it breaks and not update until it is fixed.

  2.5 Another strategy that can help with this is to have the full CouchDB
test suite run inside X, so changes to X will run the full CouchDB suite
with an updated X

Cheers
Dale


On 9 September 2015 at 19:39, Alexander Shorin <kxepal@gmail.com> wrote:

> On Wed, Sep 9, 2015 at 8:13 PM, Robert Kowalski <rok@kowalski.gd> wrote:
> >> So, what is the immediate next step that you propose to fix or
improve?
> >> What is the most squeaky wheel, or the most modest gain that we can
> make?
> >
> > I think one of the biggest problems is that we don't have a working CI
> and
> > that PRs are getting merged without running it. The other problem is
that
> > we have a lot of untested changes going into master.
> >
> > For a step by step plan I suggest:
> >
> >  - Setup a simple CI that makes it easy to test before a merge, e.g.
like
> > this [1]. The CI can be really simple, just 1 Erlang version and one OS
-
> > but let's get started somewhere. Do you have other ideas how to solve
CI,
> > PRs and the multi-repo-approach?
>
> Who will setup and host this CI machine? I guess, we don't speak about
> CI within ASF infra here.
>
> Solution of multirepo PRs is simple: use GitHub API to subscript to PR
> comments and changes. With special comment author sets references to
> other PR's. Subscription to changes is required to catch PR updates.
> Special comment is required to prevent false start when CI will try to
> build PR without related ones. Also it should be able to catch new
> references over time e.g. because author add missed tests to another
> repository after CI green build.
>
> >  - With a working CI system for the multi-repo setup merging should
just
> > happen with a green CI. Flaky tests should get fixed or deleted.
>
> Need to define policy what to do if CI is red by a reason not related to
> PR.
>
> >  - We would require that every change needs tests if not covered
> elsewhere
> > already. This might include that untestable code has to be refactored
> > first, a problem many legacy applications face (btw a good book on this
> is
> > "Working Effectively with Legacy Code". This would automatically make
our
> > code more testable and increase test coverage over time in a
> > very efficient way. This would also allow us to add features while
slowly
> > refactoring the code.
>
> What tools will be used to make this requirement mandatory?
>
> And the final note:  what is due to date (approximately) for all of this?
>
>
> While CI is a dream, what we can do to tolerate this problem *now*?
> For instance we can:
> - Reviewer who said +1 must first run full tests locally and ensure they
> are ok;
> - Reviewer must ensure that tests to the related changes are included;
> and so on. Anything else?
>
> --
> ,,,^..^,,,
>



Mime
  • Unnamed multipart/related (inline, None, 0 bytes)
View raw message