cordova-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Gill <>
Subject Re: Refactoring the CLI tests
Date Thu, 07 Nov 2013 18:02:46 GMT
I don't think we should scrap the current tests. I am totally in favor of having new end to
end tests. We need to catch regressions better. 

Braden, let me know how I can help. 

> On Nov 7, 2013, at 9:02 AM, Michal Mocny <> wrote:
> Discussing locally with Braden.. these tests seem to be testing internal
> details instead of expected functionality.  Its quite common for valid
> patches to break the tests and invalid patches to leave the tests passing.
> There have been several occurrences recently where things landed even
> though "cordova create" or "plugin add" were hosed, which seems fairly
> fundamental to keep working ;)
>> On Thu, Nov 7, 2013 at 11:57 AM, Michal Mocny <> wrote:
>> +1 to testing end-to-end, but I thought thats what BuildBot/Medic does?
>> Likely we want to ship those end-to-end test scripts so users can test
>> changes locally, but I think the tests are are inside CLI now are meant to
>> be unit tests, which yes, have very limited usefulness in isolation, but
>> perhaps shouldn't be entirely replaced?  If they really are useless (not
>> sure) then they should be fixed/removed.
>> On Thu, Nov 7, 2013 at 11:40 AM, Braden Shepherdson <>wrote:
>>> The CLI tests are bad. I propose making them better.
>>> The tests are bad for two reasons:
>>> 1. They're fragile because the tests depend on exactly the right functions
>>> being called, sometimes in the right order.
>>> 2. They don't test what we really want, which is that projects get created
>>> and all the files are in the right places.
>>> I propose letting the tests actually run filesystem and related calls,
>>> instead of always mocking them out. In the simplest form, that means
>>> running them on the real filesystem. If that's too slow, we can
>>> investigate
>>> other alternatives, like using a ramdisk, or using that emulated fs that
>>> runs everything in RAM inside Node.
>>> Mark and I are planning to work on this, starting with the CLI tests. The
>>> Plugman tests are better in this regard, but could probably stand to be
>>> really called as well.
>>> Any thoughts or objections?
>>> Braden

View raw message