cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will Stevens <wstev...@cloudops.com>
Subject Re: [PROPOSAL] Minimum Viable CI Integration
Date Sat, 05 Mar 2016 13:09:03 GMT
The choice of Golang was made for a couple reasons:
- It is the easiest language to produce a standalone runnable binary that
can fit into any heterogeneous environment.  This would be kicked off the
CI run (hopefully), and each CI environment will be installed and
configured differently, so I was trying to reduce the friction points for
people being able to adopt it once it was ready.
- I enjoy working with Golang, so this encourages me to work on this in my
spare time.  :)
- Golang is good at this type of software.

As for the proxy.  I would have to put some more thought into that.
Assuming the proxy can forward on the requests to Github and handle sending
the responses back, then we should be able to do a basic integration.  You
may be limited to only commenting with a summary and not including public
files, unless we can make the proxy work for uploading log files to an
object store as well.  Once I get a little deeper into this I can send you
a binary that you can test and we will work out the issues from there.

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Sat, Mar 5, 2016 at 1:38 AM, ilya <ilya.mailing.lists@gmail.com> wrote:

> I see where Daan is coming from :)  I thought this would be 4th, not
> exactly 7ths.
>
> I'm not against golang by any means (if anything - its my next "go" to
> language these days).
>
> Things to consider:
>
> Would notify-pr support proxy? I've been thinking on ways of
> contributing test runs, there would have to be few things i'd need to do.
>
> 1) massage the log content - such that no environment details are
> exposed, i can probably handle this with sed/awk..
>
> 2) i'm behind multiple firewalls with no internet access. however, some
> lab environments might have a proxy, so it would be nice to have a
> support for it.
>
> Thanks
> ilya
>
>
>
> On 3/4/16 6:56 AM, Will Stevens wrote:
> > Yes, I have most of it already built and will be releasing it later today
> > or over the weekend.  The reason I chose Golang is because it can be
> cross
> > compiled to be run on any system and distributed as a single binary with
> no
> > dependencies.  This means that no one will have to worry about building
> it
> > or having to change their environment at all in order to use it.  I am
> > trying to lower the barrier to entry and make it as easy as possible for
> > people to contribute back their CI execution details.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > *CloudOps* *| *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > w cloudops.com *|* tw @CloudOps_
> >
> > On Fri, Mar 4, 2016 at 8:53 AM, Daan Hoogland <daan.hoogland@gmail.com>
> > wrote:
> >
> >> Will, Do you have an implementation of notify-pr? I am asking as you
> >> specify it will be implemented in golang which seems odd. It is not
> amongst
> >> the 7 or so languages already in use.
> >>
> >> On Fri, Mar 4, 2016 at 1:54 AM, Will Stevens <williamstevens@gmail.com>
> >> wrote:
> >>
> >>> Hey Everyone,
> >>> As I am sure most of you are aware, I have been focusing a lot on ways
> to
> >>> get CI integrated back into the community.
> >>>
> >>> Today I build a little POC to validate some ideas and get a feel for a
> >>> potential approach for getting CI integrated into the Github pull
> request
> >>> workflow.
> >>>
> >>> There are multiple individuals/companies focusing on CI right now
> (which
> >>> is a good thing), but there has not really been much discussion (that I
> >> am
> >>> aware of) for how these different CI runs will push back results to the
> >>> community.  I want to make sure that nobody's work on this topic goes
> to
> >>> waste, so my goal is to provide a simple and consistent way for
> everyone
> >> to
> >>> push their results back to the community.
> >>>
> >>> Here is the basic idea (please give feedback):
> >>> - A simple cross platform command line tool with zero dependencies will
> >> be
> >>> provided (and open sourced) which will handle the submission of the CI
> >>> results back to the community.  It is written in Golang and is
> currently
> >>> called 'notify_pr'.
> >>> - At the end of a CI execution, the CI should automate the execution of
> >>> this tool to handle updating the appropriate PR with the results.
> >>>
> >>> Configuration can be done via the command line or through an INI file.
> >>> Here is an example of the configuration details.  The commit is the
> >>> commit that the CI just executed against.
> >>>
> >>> token  = <your github token>
> >>> owner  = apache
> >>> repo   = cloudstack
> >>> commit = c8443496d3fad78a4b1a48a0992ce545bde299e8
> >>>
> >>> summary_file = <a text file summary of the run>
> >>> full_detail_dir = <a directory structure to be recursively uploaded to
> >>> object store>
> >>> full_detail_files = <a comma separated list of files to upload to
> object
> >>> store>
> >>> store_api = <swift or s3>
> >>> store_endpoint = <url endpoint>
> >>> store_identity = <keystone identity or aws access key>
> >>> store_secret = <keystone password or aws secret key>
> >>>
> >>> I have not yet implemented the object storage endpoints, but I have
> code
> >>> to do it from a different project, so I just need to add it.  I will be
> >>> able to host my CI output in a swift object store, but others may need
> to
> >>> use AWS S3.  Maybe we can get sponsorship for this storage.  We will
> only
> >>> keep the logs for a window of like a week or so on the object store so
> >> the
> >>> storage usage will not be ever growing.
> >>>
> >>> Basically, the tool takes the details of the repository you are
> >> validating
> >>> a Pull Request for and the commit you are building.  It also takes the
> >>> files you would like to push to the pull request.  The summary file
> will
> >> be
> >>> shown as text in the pull request comment and the other files will be
> >>> uploaded to an object store and will be publically available for a
> period
> >>> of time (probably about a week and then get cleaned up, details TBD).
> >> The
> >>> files to be uploaded to object store could be either specified as
> >>> individual files, OR a target directory and all the files in that
> >> directory
> >>> will be recursively uploaded to the object store.
> >>>
> >>> When the tool is run, it will scan through all the open pull requests
> in
> >>> the target repository and when it finds the pull request corresponding
> to
> >>> the commit in question, it will post the details as a comment to that
> >> pull
> >>> request.  This functionality is currently working (see the attached
> >>> screenshot).  I can change the formatting and such, this is just an
> >> example.
> >>>
> >>> This is still a very rough concept that I have only worked on for a
> day,
> >>> but hopefully you guys agree that it is a good start towards a
> consistent
> >>> feedback mechanism for the community to take advantage of the different
> >>> distributed CI installations.
> >>>
> >>> Please voice your feedback and concerns.  I am sure I have not thought
> of
> >>> everything and we may still want to make fundamental changes to the
> >>> approach, but I think the concept is solid.
> >>>
> >>> Cheers,
> >>>
> >>> Will
> >>>
> >>
> >>
> >>
> >> --
> >> Daan
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message