openwhisk-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ben Browning <bbrow...@redhat.com>
Subject Re: Let's maintain and test our Swagger spec
Date Fri, 06 Jul 2018 01:20:29 GMT
After several failed attempts, I have an approach that I believe will
work well enough for our needs. I pushed some in-progress code and
swagger spec fixes to a branch on a fork just so the general approach
can be shared -
https://github.com/projectodd/incubator-openwhisk/commit/5adb6be27188dcdc5b9519b57c11802a431b3e9c

This approach uses the swagger-request-validator Java library. I'm
wiring that up to validate all requests and responses from the
WskRest* tests transparently, which you can see in
https://github.com/projectodd/incubator-openwhisk/commit/5adb6be27188dcdc5b9519b57c11802a431b3e9c#diff-5e514c6c3c27b9210d85a08d4ed3ed35
(with some cruft and debug output that I'll clean up before submitting
a PR)

This means that almost all of those tests are now failing due to
failed swagger spec validations. For now I'm going to concentrate on
making the spec match the reality and try to get all the existing
tests green with the validation in place and submit a PR. After that,
we can patch up any known but uncaught holes by increasing the test
coverage.

Ben



On Thu, Jun 28, 2018 at 11:23 PM, Carlos Santana <csantana23@gmail.com> wrote:
> Thanks Ben for the quick update on this task
>
> -cs
>
> On Thu, Jun 28, 2018 at 5:15 PM Ben Browning <bbrownin@redhat.com> wrote:
>
>> After doing some preliminary poking at this, I believe we'll want to
>> use either a tool like https://github.com/google/oatts to generate a
>> test suite from our Swagger spec OR use swagger-codegen to generate a
>> Scala client from our Swagger spec and try to plug that into the
>> existing WskRest tests.
>>
>> Using the oatts tool doesn't really fit well with the test setup in
>> the existing incubator-openwhisk repo (where the API spec lives)
>> because that generates Node.js tests.
>>
>> So, I'm leaning towards the second option, which is wiring in
>> generation of a Scala client into the gradle build and having the
>> current WskRest test client use this generated Scala client for
>> testing instead of directly invoking URLs.
>>
>> However, last time I played with Scala code generated from Swagger
>> specs it wasn't that usable. So, a bit more experimentation will
>> validate whether this option is viable or if other alternatives need
>> to be considered. I already have a handful of bugs in the API spec
>> that need to be fixed but I'm waiting to fix and push those until I
>> can get some kind of testing wired up to reproduce the bugs and verify
>> the fix.
>>
>> Ben
>>
>>
>> On Thu, Jun 21, 2018 at 3:04 PM, Carlos Santana <csantana23@gmail.com>
>> wrote:
>> > Thanks Ben for looking into this, having a good API doc/spec and matching
>> > tests is very need it.
>> >
>> > +1
>> >
>> > -cs
>> >
>> > On Thu, Jun 21, 2018 at 2:25 PM Ben Browning <bbrownin@redhat.com>
>> wrote:
>> >
>> >> Our Swagger spec
>> >> (
>> >>
>> https://github.com/apache/incubator-openwhisk/blob/92a64c291156a2cd3d6b304babc2a193a46d0699/core/controller/src/main/resources/apiv1swagger.json
>> >> )
>> >> is incomplete and doesn't accurately reflect the actual Controller
>> >> API. It's manually updated without a full test suite which means it's
>> >> easy for changes in the API to happen without the spec getting
>> >> updated.
>> >>
>> >> An accurate Swagger spec will not only better document the OpenWhisk
>> >> API but also allow autogenerated clients in multiple languages to
>> >> supplement or eventually replace some of the existing client
>> >> implementations we have today. It also paves way for future compatible
>> >> server implementations, whether they be rewrites of the existing
>> >> Controller or stub test harnesses to facilitate end-to-end testing on
>> >> a developer's laptop.
>> >>
>> >> As I'm already working with autogenerating code from the Swagger spec
>> >> for other purposes, I'm happy to take the lead on this effort. I'd
>> >> like to take a two-pronged approach for a test suite:
>> >>
>> >> * Generate a server stub from the spec and ensure the wsk CLI can
>> >> communicate with it.
>> >>
>> >> * Generate a client stub from the spec and ensure it can communicate
>> >> with the existing API.
>> >>
>> >> There are a lot of details to figure out from those two statements.
>> >> And, this approach won't guarantee 100% correctness of the spec. The
>> >> only way to do that would be to generate all supported clients and the
>> >> Controller API from the spec. But, this should get us started in the
>> >> right direction.
>> >>
>> >> If anyone's gone down this path before and has some wisdom to share,
>> >> please speak up!
>> >>
>> >> Thanks,
>> >>
>> >> Ben
>> >>
>>

Mime
View raw message