flex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alex Harui <aha...@adobe.com>
Subject Re: Build failed in Jenkins: flex-sdk_mustella #119
Date Sun, 02 Jun 2013 21:59:53 GMT

On 6/2/13 10:18 AM, "Erik de Bruin" <erik@ixsoftware.nl> wrote:

>Does that imply that once you cut the new baselines ALL tests will
>pass? That sounds cool! I've never seen a fully successful run of
>Mustella ;-)
I've only seen about 4.  It looks like you are still getting some
timeouts.  But for me on the Mac and Win on Friday, running -all followed
by -failures ended up with 0 failures, but I realized after that I hadn't
pulled changes since earlier that week.

So, the metric for success may require that script that checks for
failures.txt and runs with the -failures options.

>Once you get all tests to pass I'll restart the twice daily schedule
>and we will finally have a max of 12 hours between commit and a signal
>something broke the SDK. Let the contributions start streaming in!
>Also, this should help us a long way towards 4.10, shouldn't it?
Yup, and I'll get the patch tester up and running early next week.  We had
both at Adobe.  The patch server took a .patch file, figured out the
minimum set of tests to run, ran them, and sent you back results.  That
has the advantage of
1) You find out in private that you broke something
2) You should find out sooner than 12 hours, although a change to
TextBase.as took 4 hours.
3) If you submit a patch and it breaks something, you don't have to
scramble to revert your change, since all future changes are going to
break until the bad change is found an reverted.  The patch server reverts
the patch after the run so there isn't cross-pollinization.
4) You know for sure it was you.  At 12 hour intervals, several changes
from several folks get tested at once.
5) Non-committers can test their patches before submitting.

But for sure we need a commits-tester, which is what I call what you've
set up since the patch-tester is optional.  We can't force folks to use it
and the only thing that really matters is whether the code in the repo
passes or not.

But the interesting thing for me once we get this Jenkins thing running
is: if several committers also set up this same Jenkins job, could we have
them each run a subset of the mustella tests and report much sooner?  I
think it was you who suggested we could create a massively parallel
mustella system.


View raw message