couchdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dale Harvey <d...@arandomurl.com>
Subject Re: [PROPOSAL] Improving the quality of CouchDB
Date Thu, 10 Sep 2015 09:53:23 GMT
And if anyone wants to make a start on travis

https://github.com/pouchdb/pouchdb/blob/master/.travis.yml and
https://github.com/pouchdb/pouchdb/blob/master/bin/run-couch-master-on-travis.sh

Should get you most of the way there

On 10 September 2015 at 11:40, Dale Harvey <dale@arandomurl.com> wrote:

> I dont think CI is a dream, It should take an afternoon at most to get
> CouchDB setup on travis on a single platform to ensure no major regressions
> come though. If anyone wants help doing that feel free to ping me on
> #pouchdb / #couchdb, we already test couchdb master in travis.
>
> I do think multirepos is an issue and that solution is not so simple, we
> went through the same issues with pouchdb as we attempted to split out our
> repository.
>
> The basic problem is, X is a downstream component of CouchDB in its own
> repo, X may have its own unit tests etc, however I can still make a change
> to X, have it all passing but it can still complete break the CouchDB
> suite. This also entirely break git bisect so not only are breakages more
> likely but they are significantly harder to debug.
>
> The 2 main solutions to that work to some degree for us
>
> 1. Dont split out into lots of repositories, if you put those components
> inside the CouchDB repo, then they will get the CouchDB tests run against
> them when changes are made and you wont break the CouchDB repo.
>
> 2. Anything that does live outside the CouchDB repo, pin their version
> inside the CouchDB repo, dont have commits to subproject X automatically be
> applied to CouchDB, that means you can commit whatever you want to X but
> CouchDB will still be working, when you come to update the version of X you
> pinned, you can see that it breaks and not update until it is fixed.
>
>   2.5 Another strategy that can help with this is to have the full CouchDB
> test suite run inside X, so changes to X will run the full CouchDB suite
> with an updated X
>
> Cheers
> Dale
>
>
> On 9 September 2015 at 19:39, Alexander Shorin <kxepal@gmail.com> wrote:
>
>> On Wed, Sep 9, 2015 at 8:13 PM, Robert Kowalski <rok@kowalski.gd> wrote:
>> >> So, what is the immediate next step that you propose to fix or improve?
>> >> What is the most squeaky wheel, or the most modest gain that we can
>> make?
>> >
>> > I think one of the biggest problems is that we don't have a working CI
>> and
>> > that PRs are getting merged without running it. The other problem is
>> that
>> > we have a lot of untested changes going into master.
>> >
>> > For a step by step plan I suggest:
>> >
>> >  - Setup a simple CI that makes it easy to test before a merge, e.g.
>> like
>> > this [1]. The CI can be really simple, just 1 Erlang version and one OS
>> -
>> > but let's get started somewhere. Do you have other ideas how to solve
>> CI,
>> > PRs and the multi-repo-approach?
>>
>> Who will setup and host this CI machine? I guess, we don't speak about
>> CI within ASF infra here.
>>
>> Solution of multirepo PRs is simple: use GitHub API to subscript to PR
>> comments and changes. With special comment author sets references to
>> other PR's. Subscription to changes is required to catch PR updates.
>> Special comment is required to prevent false start when CI will try to
>> build PR without related ones. Also it should be able to catch new
>> references over time e.g. because author add missed tests to another
>> repository after CI green build.
>>
>> >  - With a working CI system for the multi-repo setup merging should just
>> > happen with a green CI. Flaky tests should get fixed or deleted.
>>
>> Need to define policy what to do if CI is red by a reason not related to
>> PR.
>>
>> >  - We would require that every change needs tests if not covered
>> elsewhere
>> > already. This might include that untestable code has to be refactored
>> > first, a problem many legacy applications face (btw a good book on this
>> is
>> > "Working Effectively with Legacy Code". This would automatically make
>> our
>> > code more testable and increase test coverage over time in a
>> > very efficient way. This would also allow us to add features while
>> slowly
>> > refactoring the code.
>>
>> What tools will be used to make this requirement mandatory?
>>
>> And the final note:  what is due to date (approximately) for all of this?
>>
>>
>> While CI is a dream, what we can do to tolerate this problem *now*?
>> For instance we can:
>> - Reviewer who said +1 must first run full tests locally and ensure they
>> are ok;
>> - Reviewer must ensure that tests to the related changes are included;
>> and so on. Anything else?
>>
>> --
>> ,,,^..^,,,
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message