cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Filip Maj <...@adobe.com>
Subject Re: Medic - Automated Testing
Date Tue, 30 Jul 2013 16:45:17 GMT
That all sounds great. Kick it off David!

On 7/30/13 5:43 AM, "David Kemp" <drkemp@google.com> wrote:

>It looked like much of the device-specific deployment bits could be
>applied
>between cli prepare and cli run - as long as run does not do a prepare -
>or
>a deploy option is added (discussed in a separate thread).
>Then I think you can deploy to a device and start a timer (x n devices).
>
>There's probably stuff I'm missing, but the lack of a deploy-only option
>in
>cli was the only roadblock I saw.
>
>I was thinking of a flow like the 'working with 3.0' page with a few
>additions. That keeps the test environment really close to our recommended
>workflow.
>
>Roughly:
>* coho to get everything (platforms, plugins, js, cli, mobilespec)
>* npm install cli
>* npm install js
>* build js (log tests)
>* patch json
>* cli to add platforms, plugins
>* copy in the newest js
>* hook up the mobilespec project
>* cli prepare
>* Medic bits to patch the project for each platform
>* cli to compile for each platform
>* cli to deploy to each device
>
>all along there, the Medic/couchDB stuff needs to be recording
>success/failures.
>
>
>
>
>
>
>
>On Mon, Jul 29, 2013 at 6:04 PM, Filip Maj <fil@adobe.com> wrote:
>
>> Definitely agree that the presentation bits need a ton of work. I
>> bikeshedded the whole thing. Essentially the web frontend can be
>> completely tossed. The reusable bits are the bits integrating with the
>> couch db.
>>
>> I really want to have medic run on top of cordova-cli, but there are
>> limitations there. A bunch of the multi-device deployment scripts in
>>medic
>> are built right on top of native mobile tooling, and I'm not sure if
>>that
>> is possible to implement right now on top of cordova-cli. Additionally,
>>I
>> am unsure if that is the type of use case that we want to support in
>> cordova-cli. If so: we should refactor cordova-cli to enable it. If not,
>> well, then we support it in medic.
>>
>> Once the repo is up and we push stuff from my fork over we can start
>> figure out a roadmap for it. We DEFINITELY should talk about medic at
>>our
>> committer meeting coming up .. Whenever that is.
>>
>> On 7/29/13 11:35 AM, "David Kemp" <drkemp@google.com> wrote:
>>
>> >I have been reviewing the medic components and the procedures for
>>testing
>> >the various parts of the Cordova project. With the changes in 3.0,
>>there
>> >are plugins that need to be handled and as already mentioned, I would
>>like
>> >to expand to include the tooling as well. One side effect of this is
>>error
>> >reporting on about 20 'projects' instead of just 2-3.
>> >
>> >Currently medic does a bunch of things :
>> >* provides a dashboard of the test results and commits
>> >* watches for commits on the android, ios,blackberry and mobilespec
>> >projects
>> >* when commits happen
>> >  * get the newest code
>> >  * build and report any errors to couchdb
>> >  * modify a few things to add jasmine reporting to a couchdb
>> >  * deploy to a bunch of appropriate devices
>> >  * log to couchdb if the tests do not finish in time
>> >
>> >With the 3.0 structure, the git monitoring and project structure gets
>> >quite
>> >a bit more interesting. Also we have now added a bunch of tools that do
>> >some of this as part of their flow that we should be testing.
>> >
>> >So ....
>> >
>> >I propose that the cordova-cli and cordova-coho tools be used as part
>>of
>> >the test process. This means using something to watch for commits, then
>> >launch coho to get a full set of plugins, platforms and mobilespec. It
>> >might be wise at this point to even compile cordova-js and log the
>>results
>> >of the tests there.
>> >Then some medic magic needs to happen to prepare the project for
>>jasmine
>> >reporting and auto-launch. Then use cli to build and deploy.
>> >Any errors in the tooling would need to be captured in the test DB as
>> >separate items from the errors in the platforms.
>> >
>> >For this to work, the medic components to manage the db and prepare a
>> >project for an automated mobilespec would need to be broken out into
>> >separate utilities. Also there would probably need to be a few
>>additions
>> >to
>> >cli to target multiple devices, etc.
>> >
>> >At the top level, we still need discovery of new commits, but over a
>>broad
>> >range of projects. There are bits in medic that do that (needs a little
>> >work) or possibly use buildbot for that layer. One advantage of that
>>would
>> >be the already built-in notification for a broken build (email/irc,
>>...).
>> >Perhaps we could even report the commit(s) that broke it...
>> >
>> >The data presentation/reporting will need some work. There will be lots
>> >more to show.
>> >
>> >Thoughts and/or discussion?
>>
>>


Mime
View raw message