incubator-flex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Conrad <jeff...@gmail.com>
Subject Re: [RT] Ideas on getting the entire test suite for the sdk to run in 10 minutes or less
Date Tue, 14 Aug 2012 16:23:09 GMT
I want testing to be as painless and as reliable as possible.

Perhaps the IaaS route is a good first step.  If we can get the suite
running in 10 minutes or less on an IaaS grid, that's a lot of runs that
could happen.  In addition, if we ever move to Git and use Gerrit for code
reviews, there's a Jenkins plugin that will run builds based on any given
changeset, which would effectively let anyone do it by way of submitting to
the code review.  https://wiki.jenkins-ci.org/display/JENKINS/Gerrit+Plugin

Is there a Jenkins plugin to apply an attached patch to a JIRA issue?  I
did a quick scan and didn't see one.  There is a JIRA plugin that can
update JIRA issues based on issue ID's mentioned in commit comments,
though. https://wiki.jenkins-ci.org/display/JENKINS/JIRA+Plugin

Also, what's 'hurricane'?  I saw it mentioned in the mustella source
yesterday, but the word hurricane is too generic for Google to be of any
help.  Every phrase I thought of didn't turn up anything useful to answer
that question on my own.

Jeff

On Tue, Aug 14, 2012 at 12:10 PM, Peter Ent <pent@adobe.com> wrote:

> Hi Jeff,
>
> You are suggesting that if someone has the horsepower available (i.e.,
> access to an IaaS cloud), the mustella tests should run, in their entirety
> in 10 mins or less - versus - anyone being able to do this, correct? I
> think I'm reading more into your 10-minutes-or-less suggestion than
> intended.
>
> I do think the tests can be broken down into better units as they are all
> kind of lumped together, so that's a particularly good goal.
>
> --peter
>
> On 8/14/12 12:01 PM, "Jeff Conrad" <jeffxpx@gmail.com> wrote:
>
> >Hi,
> >
> >I'd like to help the project get to a point to where we can run the entire
> >test suite for the sdk in 10 minutes or less.  I think that's a worthy
> >goal, and I'm willing to help make that a reality.
> >
> >If we get the testing time down to being that fast, that will mean we can
> >review a lot more contributions, and it could also mean that potential
> >contributors can submit patches that they know don't fail any tests before
> >submitting the patch.  It also means that if one of those tests fail, the
> >person writing the code still has the context of what they did fresh in
> >their mind and can fix it quickly.
> >
> >For reference, I ran the entire Mustella suite last night, and it took 4
> >hours, 3 minutes, and 50 seconds to run ./mini_run.sh -createImages -all
> >last night on a quad-core machine running Windows 7 using the Git Bash
> >(yay! no cygwin!).  To make it work with the Git Bash, I changed the shell
> >variable in mustella/build.xml to just sh.exe.  As a plus, shellrunner.sh
> >somewhat intelligently parallel-ized the compilation of all the test swfs
> >so it was compiling 4 swfs at a time, and then the ant script ran all the
> >test swfs one at a time.  I know mustella is more a functional /
> >integration test suite, so by definition it's going to run slower than a
> >suite of unit tests.
> >
> >I also know that at least a few people on this list want to refactor the
> >sdk so it's unit-testable.  I definitely support this and would like to
> >help in any way I can on that front.
> >
> >I want the entire suite: unit, integration, and functional to be able to
> >run in 10 minutes or less.  I have two ideas as to how we can make this
> >happen, and I'm open to more.
> >
> >One idea would be to intelligently look at the files that a given patch /
> >changeset affects and only build and run the tests that test that
> >functionality.
> >
> >The other idea that comes to mind is spin up a group of virtual servers in
> >an IaaS cloud and split up the workload.  If we can evenly split this up,
> >and my original number is right, it means 24+ servers.  There are easy
> >ways
> >to get the files across that many servers quickly, so the file transfer
> >part wouldn't be that hard.
> >
> >What do you guys think?  Where would you start?
> >
> >Jeff
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message