asterixdb-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Maxon <ima...@uci.edu>
Subject Re: Proposal for alteration of criteria for automatic patch verification
Date Thu, 14 Jan 2016 23:07:38 GMT
We can definitely have a job that will take a commit as a parameter that is
run simply on demand rather than via any sort of trigger or timer. I have
had a job for this actually at various points but I don't know if the ones
I have right now are pointed at the correct repositories.


For the time being, to start impelementing this since there seems to be no
objection, I have set up a job to run on each commit. If it looks like it's
working good after the next commit I will change AsterixDB to use 'mvn
test' instead of 'mvn verify' for automatic verification.

-Ian

On Wed, Jan 13, 2016 at 4:27 PM, Till Westmann <tillw@apache.org> wrote:

> I think that to find out quickly which change introduces the issue, it
> would be nice if there was an easy way to have Jenkins run the “long suite”
> for a specific commit. That way we could identify the problematic commit in
> the standardized environment and without relying on developer setup.
>
>
> On 13 Jan 2016, at 16:20, Mike Carey wrote:
>
> Agreed.  I'd vote for making sure that the broken daily test report be
>> accompanied by a list of the changes that occurred since the last
>> successful run - and that it be the responsibility of all folks with
>> changes on that list to gang up and ensure that the problem is identified
>> and resolved in a timely fashion.  We should probably spell out what that
>> looks like, but it seems eminently reasonable....
>>
>> On 1/13/16 4:03 PM, Chris Hillery wrote:
>>
>>> +1 for Ian's initial proposal. This kind of pre-commit validation is
>>> always
>>> a balancing act between being comprehensive and being timely, and I think
>>> sticking with the distinction between "mvn test" and "mvn verify" is a
>>> clean and simple way to divide the tests.
>>>
>>> Given that pushing the Gerrit head to ASF is manual anyway, it wouldn't
>>> be
>>> TOO much headache to also enact Ildar's suggestion - basically, whoever
>>> does the ASF push would need to wait to ensure each commit passes the
>>> corresponding daily tests. However, IMHO at least this is probably not
>>> necessary. If a bug is uncovered in the daily tests, we'll need to push
>>> an
>>> additional commit to fix it, and that additional commit (plus the
>>> original
>>> buggy commit) will still end up on ASF anyway. And I worry a bit about
>>> anything that would cause the ASF Github mirror to stay out of sync with
>>> Gerrit for very long.
>>>
>>> Ceej
>>> aka Chris Hillery
>>>
>>> On Wed, Jan 13, 2016 at 3:42 PM, Ildar Absalyamov <
>>> ildar.absalyamov@gmail.com> wrote:
>>>
>>> It would be really nice to break Jenkins test build into two stage
>>>> process: ‘mvn test’ on each patch set submission and ‘mvn verify’
after
>>>> the
>>>> patch received +2 and got merged into Gerrit, BUT before it got merged
>>>> to
>>>> ASF repo. Although I am not sure what should be done in a case when the
>>>> latter build does not pass, since it’s already merged into Gerrit and we
>>>> cannot revert that.
>>>>
>>>> On Jan 13, 2016, at 15:34, Ian Maxon <imaxon@uci.edu> wrote:
>>>>>
>>>>> Hi all,
>>>>> I'd like to field the idea of changing the criteria we use to allow
>>>>>
>>>> Jenkins
>>>>
>>>>> to give +1 verified on patch sets. Right now, it runs 'mvn verify',
>>>>> which
>>>>> runs darn near about every test we have. While this was somewhat OK in
>>>>>
>>>> the
>>>>
>>>>> past, I think now, and going forward, it takes a long time, and it will
>>>>> only take longer as we make the integration tests more thorough.
>>>>>
>>>>> Additionally, I'd like to expand and re-enable the automated
>>>>>
>>>> vagrant-based
>>>>
>>>>> cluster integration tests, and unfortunately there isn't a simple way
>>>>> to
>>>>> make these work inside a docker container. I can get it to work by
>>>>> hand,
>>>>> but to make it automated I would have to fork or submit a patch to the
>>>>> Jenkins Docker cloud plugin, and it might not be a small or
>>>>> particularly
>>>>> clean/easy patch either.
>>>>>
>>>>> I'd like to change the goal that's run in the automatic tests on every
>>>>> commit to 'mvn test' rather than 'mvn verify'. This will still run all
>>>>> of
>>>>> the ExecutionTest(s), and the optimizer tests, and so on. It will stop
>>>>>
>>>> the
>>>>
>>>>> recovery tests and asterix installer tests from being run every time,
>>>>> however.
>>>>>
>>>>> However these tests are still certainly valuable, so I'd run them on
a
>>>>> timer, perhaps daily, so that we would still catch bugs that these
>>>>> tests
>>>>> reveal rather promptly. Due to the fact we only commit a single digit
>>>>> number of times to master each day generally, the granularity of which
>>>>> commit will have introduced a bug will hopefully be rather low.
>>>>>
>>>>> Thoughts/comments?
>>>>>
>>>>> Thanks,
>>>>> -Ian
>>>>>
>>>> Best regards,
>>>> Ildar
>>>>
>>>>
>>>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message