Mailing-List: contact dev-help@cordova.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cordova.apache.org
Received-SPF: pass (athena.apache.org: domain of agrieve@google.com designates
 209.85.192.182 as permitted sender)
MIME-Version: 1.0
Sender: agrieve@google.com
In-Reply-To: 
 <CAMTxO40+VQg0Eh59SzDCGU3354d=vN=W4Ua5znGb7Z=iNFV3Wg@mail.gmail.com>
References: 
 <CABiQX1Uac5pD1tcY2RD7CKrJhys5AmtPFmf+UwUPaXNe9tbWnw@mail.gmail.com>
 <461A1ADB-C3A4-4833-A01A-B3D844A40C31@gmail.com>
 <CAOe3-e4-tPnNo1y1EnOjsTbYwC0DNLsJ0ha5pE=14A1Sq8LwJg@mail.gmail.com>
 <CAMTxO437j7UBCMfwEVzJWYM4tM4-f-XwMoH9AWWdJHh1LSie9Q@mail.gmail.com>
 <CAMTxO40+VQg0Eh59SzDCGU3354d=vN=W4Ua5znGb7Z=iNFV3Wg@mail.gmail.com>
From: Andrew Grieve <agrieve@chromium.org>
Date: Thu, 25 Jul 2013 14:07:49 -0400
Message-ID: 
 <CABiQX1XS7c+kRHDVNS9Js0X9+TFk=xhf6PzZ=UaWM1rHW8kW5w@mail.gmail.com>
Subject: Re: medic
To: dev <dev@cordova.apache.org>
Cc: Mike Billau <mike.billau@gmail.com>, Marcel Kinard <cmarcelk@gmail.com>,
 drkemp@chromium.org
Content-Type: multipart/alternative; boundary=047d7b15afe59ce41f04e259e8cf

--047d7b15afe59ce41f04e259e8cf
Content-Type: text/plain; charset=ISO-8859-1

+dev


On Thu, Jul 25, 2013 at 2:03 PM, David Kemp <drkemp@google.com> wrote:

> Hi all,
>
> In the spirit of trying to share the goods, I would like to see if we can
> co-ordinate some of the efforts, and more importantly some of the data.
> With the changes in 3.0, I don't think the data structure used previously
> is sufficient.
>
> I would like to see results that:
> * for mobilespec: include the platform, platform version, js version,
> plugin list (with versions)
> * include results for cordova-cli
> * include results for cordova-coho
> * include results for cordova-js tests
> * include results for other tools that are testable
>
> The current medic structure tags tests with <platform><sha>.
>
> A possible solution is to add an additional dataset that ties a Test_ID to
> a test description. The test description could be a a structured object
> indicating the list of <component><sha> that was in the test. If we expand
> the <platform> to mean <component>, then the test results get saved with a
> tag of <component><test_id>.
>
> So you would get a test like:
> test_id=ANDROID_1234
> results="270 passed, 0 failed"
> components="
> cordova-android: 980c4699b0993b1718c2f0a1761f7d3cc49a0c9b
> cordova-cli: 1fe77b0be1997c42da31606cdbca9eb9e90a954e
> cordova-js: d716e31b39bd39a06f1db80040e2c8a6a78a34e8
> cordova-mobile-spec: f5c0a54d056c6e9b76f59eac2b2908ddbfed3e65
> cordova-plugin-battery-status: f52824aba6aa0f470d2ee5b3d53466fd8668ac56
> cordova-plugin-camera: 5d747c739ddd41b731f77fb531481a709ea497cd
> cordova-plugin-console: 3cb1165a3f95e86045e9800ba3e8b4ffb0ef5a91
> cordova-plugin-contacts: 5f0847d4a13a90df5b37bdcbabae6936610a5345
> cordova-plugin-device: 3c5832d2cca354bb1309eef738a848e2d4b87e3e
> cordova-plugin-device-motion: 43e653378bb3bc98ffd39c1b8d0e2a0a3d412c6c
> cordova-plugin-device-orientation: a522c97336d859a57c89d11a2dde7cb9a5771b15
> cordova-plugin-dialogs: 96c52b4d8411f7745fec5e154ad75474a4fc4d07
> cordova-plugin-file: 698d7f6d2d2f2e9364451d5c0b4c763eae97c192
> cordova-plugin-file-transfer: e37895a56e314832d8480dfde221a1f72d915140
> cordova-plugin-geolocation: d8f8aa5beef0b16d656662d59b31a0bee1b69b0d
> cordova-plugin-globalization: 8caedd2c6674e485dd46941f3f0106f7b17f1bbf
> cordova-plugin-inappbrowser: dd7b20543f20660b4be71721e404e4ba7219aa7c
> cordova-plugin-media: 9538b8c10d01fac785b3764985e7d836046802fb
> cordova-plugin-media-capture: ba76f1121b34376fe11c58bb30ebe8ab8481281a
> cordova-plugin-network-information:
> e45286df35487a4414756d8fb93597ea454cb1ff
> cordova-plugin-splashscreen: 33f353d2e9684e189f8b7d59c066253f2fe73f48
> cordova-plugin-vibration: 5c08769e83a09e900fe301f140bbe143390e8eb0 "
>
> Better ideas anyone?
>
>
>
>
> On Fri, Jul 19, 2013 at 2:38 PM, David Kemp <drkemp@google.com> wrote:
>
>> Hi Mike,
>> So, we are on much the same road. I also had medic working briefly
>> pre-3.0.
>> When the 3.0 structure changed, I put it aside for a while since it was
>> clearly going to take a bit of effort to change medic to run with 3.0.
>>
>> Today, while the dust settles a bit I have been looking at the idea of
>> using buildbot for the primary tooling, and using coho and cordova-cli for
>> a bunch of the build steps.
>>
>> My goal was to include coho and cli in the test chain so we can detect
>> failures there too.
>>
>> I also looked at the issue of tracking versions and had not yet decided
>> on any good way to describe a test set in a similar way to the simple
>> <platform><sha>. In the case of 3.X, it seems like it should be:
>>  <platform><sha><mobile-spec><sha>[..<plugin><sha>..]
>>
>> David
>>
>>
>>
>>
>> On Fri, Jul 19, 2013 at 2:01 PM, Mike Billau <mike.billau@gmail.com>wrote:
>>
>>> Hi,
>>>
>>> Initially I was just working on understanding what medic was doing and
>>> how. It's a pretty neat piece of code, Fil did a great job. I am new to
>>> node so this step has taken a bit longer than expected.
>>>
>>> You'll notice that the code is pretty old, so at first I was just
>>> updating it to work with the 2.X stream - doing things like removing the
>>> -VERSION suffix from cordova.js file, figuring out how to write my couchdb
>>> URL into the whitelist, etc. I was able to get medic to work for a few
>>> hours with three different Android devices, it was cool.
>>>
>>> Things are a bit different now though with the 3.X stream and having
>>> everything split apart. The last few days I have been working on
>>> integrating the CLI. On one hand, using the CLI seems to simplify a lot of
>>> the code, especially with the mobile spec create script that Andrew checked
>>> in yesterday (haven't integrated this yet but seems simple enough.)  On the
>>> other hand, with everything split out I am not sure if it is going to be
>>> possible to pull down the code from a specific SHA and test it out - at
>>> least not until the medic tests themselves get added into their specific
>>> repos. (How would you know which version of mobile spec matches the current
>>> SHA? By timestamp?)  I don't really think this is an issue though because
>>> it is a CI...if we just pull the HEAD of every repo when a new commit comes
>>> in and run the test, then it should be fine. (If there is a need to test
>>> old SHA, one could build that specific version of mobile spec, run through
>>> the automated tests, and hit a "Report to DB" button or something.)
>>>
>>> My current goals are to 1. integrate Andrew's script so I can have a
>>> single node command instead of 15 nested .exec callbacks, 2. Figure out why
>>> the gitpubsub always gives me JSON errors when a new commit goes in, 3. Get
>>> it running and reporting data to the couchdb, 4. Clean up the code / remove
>>> the specific SHA part 5. Check in with Ryan Willoughby who has been working
>>> on integrating PhoneGapBuild and see if he has done any 3.0 work 6. Poke
>>> the apache infra guys to see if they have set up a couch db machine for us
>>>
>>> I have also been researching types of USB hubs to use - the Adobe guys a
>>> while back told me about their huge 42 port hub. So far I have been testing
>>> the 4 port powered Pluggable devices and they daisy chain well and can do
>>> data+power with all of our devices save Samsung tablets and some iDevices.
>>> I have been compiling a table internally of the device/hub compatibility
>>> that I'd like to share/collaborate on.
>>>
>>> Hopefully we can collaborate and get this thing running. A few days ago
>>> we talked about how it'd be cool to add a "-ci" parameter to the CLI so
>>> that other offices can easily get medic up and running with whatever
>>> devices they have. I think this would be a pretty useful thing to add.
>>>
>>> Hope ya'ller having fun at the conference!
>>>
>>> Mike
>>>
>>>

--047d7b15afe59ce41f04e259e8cf--