cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bharat Kumar <bharat.ku...@accelerite.com>
Subject Re: [PROPOSAL] Minimum Viable CI Integration
Date Tue, 08 Mar 2016 05:35:25 GMT
Hi Srinivas,

The tests get skipped because the hardware requirement for those tests is not satisfied. I
will try to add more details to
the skipped tests so that people can run manually if needed.

The details of the hardware used can be added, but I am thinking of publishing it in the wiki
rather than adding these in the test results.

Thanks for the pointers.

—Bharat.


On 08-Mar-2016, at 10:36 AM, Gandikota Srinivas <gandikota.srinivas@gmail.com<mailto:gandikota.srinivas@gmail.com>>
wrote:

Hi Bharat,

Great job, report looks really cool.
Will you be able to add few more details like the setup info (number of hosts, etc)

Report indicates few tests are skipped. is this due to setup limitations? can the skipped
tests be also listed so that some of us can run those specific tests (say manually) and post
the results.

Thanks,
Srinivas

On Mon, Mar 7, 2016 at 11:06 PM, Bharat Kumar <bharat.kumar@accelerite.com<mailto:bharat.kumar@accelerite.com>>
wrote:
Hi guys,

I am also working on the similar reporting problem, here is what i did

link to the report https://github.com/bvbharatk/cloud-stack/pull/1

I am thinking this is good enough for now, I want to start posting the results on each pr
as shown in the above link.
please give me your comments or suggestions.

Thanks,
Bharat
On 05-Mar-2016, at 7:02 PM, Will Stevens <wstevens@cloudops.com<mailto:wstevens@cloudops.com><mailto:wstevens@cloudops.com<mailto:wstevens@cloudops.com>>>
wrote:

Daan

Regarding the obligatory provider id.  I agree, but I am still trying to
figure out the details.  Creating distinct runs that have their own status
is done by setting the 'context'.  I think we would need to have two pieces
to this.  A provider id and an environment id.

So for example.  Lets assume that my provider id is 'CloudOps' and I have
two different environments, one for 'KVM' and one for 'Xen' (for example).
I would then the tool would produce the following two independent CI
statuses.
'CloudOps - KVM' : with a basic description of the environment.
'CloudOps - Xen' : again with a basic description of the env.
... and so on ...

I am still sorting out the details here as well as making it easy for us to
integrate this into the apache/cloudstack repo with the access we currently
have.  Adding this is 'no biggy' for me because I am building this tool as
we speak, and trying to tailor it to our (ACS) needs, so this type of
feedback is perfect as it allows me to adapt the tool before I get too deep.

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com<http://cloudops.com/><http://cloudops.com<http://cloudops.com/>>
*|* tw @CloudOps_

On Sat, Mar 5, 2016 at 4:17 AM, Daan Hoogland <daan.hoogland@gmail.com<mailto:daan.hoogland@gmail.com><mailto:daan.hoogland@gmail.com<mailto:daan.hoogland@gmail.com>>>
wrote:

Will

Gret work, especially the thing you are showing in link [4], I would like
to make an enhancement request and that is a obligatory provider id. Only
if it is no biggy for you!
Several people may decide to do a XVM on ChildrensOS for instance and so we
may be aware of an obscurity that is different. If one fails and the other
succeeds it is easily identified.

Ilya,

I have been playing with go and it is a very nice language for such a
simple script, though it wasn't exactly designed for it. So don't read my
comment/question as an objection. But we do have
bash,python,c#,java,javascript,xslt,sql at least. That is not counting the
build system and I am sure the hyperv has some extra windows specific
stuff.
To me it is inherent to the nature of across platform orchestration and
provisioning system so it is fine. It is something to consider. On the
other hand when bringing in new tools we don't make the choice so.... I am
ranting, I guess.



On Sat, Mar 5, 2016 at 7:38 AM, ilya <ilya.mailing.lists@gmail.com<mailto:ilya.mailing.lists@gmail.com><mailto:ilya.mailing.lists@gmail.com<mailto:ilya.mailing.lists@gmail.com>>>
wrote:

I see where Daan is coming from :)  I thought this would be 4th, not
exactly 7ths.

I'm not against golang by any means (if anything - its my next "go" to
language these days).

Things to consider:

Would notify-pr support proxy? I've been thinking on ways of
contributing test runs, there would have to be few things i'd need to do.

1) massage the log content - such that no environment details are
exposed, i can probably handle this with sed/awk..

2) i'm behind multiple firewalls with no internet access. however, some
lab environments might have a proxy, so it would be nice to have a
support for it.

Thanks
ilya



On 3/4/16 6:56 AM, Will Stevens wrote:
Yes, I have most of it already built and will be releasing it later
today
or over the weekend.  The reason I chose Golang is because it can be
cross
compiled to be run on any system and distributed as a single binary
with
no
dependencies.  This means that no one will have to worry about building
it
or having to change their environment at all in order to use it.  I am
trying to lower the barrier to entry and make it as easy as possible
for
people to contribute back their CI execution details.

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com<http://cloudops.com/><http://cloudops.com<http://cloudops.com/>>
*|* tw @CloudOps_

On Fri, Mar 4, 2016 at 8:53 AM, Daan Hoogland <daan.hoogland@gmail.com<mailto:daan.hoogland@gmail.com><mailto:daan.hoogland@gmail.com<mailto:daan.hoogland@gmail.com>>

wrote:

Will, Do you have an implementation of notify-pr? I am asking as you
specify it will be implemented in golang which seems odd. It is not
amongst
the 7 or so languages already in use.

On Fri, Mar 4, 2016 at 1:54 AM, Will Stevens <
williamstevens@gmail.com<mailto:williamstevens@gmail.com><mailto:williamstevens@gmail.com<mailto:williamstevens@gmail.com>>>
wrote:

Hey Everyone,
As I am sure most of you are aware, I have been focusing a lot on
ways
to
get CI integrated back into the community.

Today I build a little POC to validate some ideas and get a feel for
a
potential approach for getting CI integrated into the Github pull
request
workflow.

There are multiple individuals/companies focusing on CI right now
(which
is a good thing), but there has not really been much discussion
(that I
am
aware of) for how these different CI runs will push back results to
the
community.  I want to make sure that nobody's work on this topic goes
to
waste, so my goal is to provide a simple and consistent way for
everyone
to
push their results back to the community.

Here is the basic idea (please give feedback):
- A simple cross platform command line tool with zero dependencies
will
be
provided (and open sourced) which will handle the submission of the
CI
results back to the community.  It is written in Golang and is
currently
called 'notify_pr'.
- At the end of a CI execution, the CI should automate the execution
of
this tool to handle updating the appropriate PR with the results.

Configuration can be done via the command line or through an INI
file.
Here is an example of the configuration details.  The commit is the
commit that the CI just executed against.

token  = <your github token>
owner  = apache
repo   = cloudstack
commit = c8443496d3fad78a4b1a48a0992ce545bde299e8

summary_file = <a text file summary of the run>
full_detail_dir = <a directory structure to be recursively uploaded
to
object store>
full_detail_files = <a comma separated list of files to upload to
object
store>
store_api = <swift or s3>
store_endpoint = <url endpoint>
store_identity = <keystone identity or aws access key>
store_secret = <keystone password or aws secret key>

I have not yet implemented the object storage endpoints, but I have
code
to do it from a different project, so I just need to add it.  I will
be
able to host my CI output in a swift object store, but others may
need
to
use AWS S3.  Maybe we can get sponsorship for this storage.  We will
only
keep the logs for a window of like a week or so on the object store
so
the
storage usage will not be ever growing.

Basically, the tool takes the details of the repository you are
validating
a Pull Request for and the commit you are building.  It also takes
the
files you would like to push to the pull request.  The summary file
will
be
shown as text in the pull request comment and the other files will be
uploaded to an object store and will be publically available for a
period
of time (probably about a week and then get cleaned up, details TBD).
The
files to be uploaded to object store could be either specified as
individual files, OR a target directory and all the files in that
directory
will be recursively uploaded to the object store.

When the tool is run, it will scan through all the open pull requests
in
the target repository and when it finds the pull request
corresponding
to
the commit in question, it will post the details as a comment to that
pull
request.  This functionality is currently working (see the attached
screenshot).  I can change the formatting and such, this is just an
example.

This is still a very rough concept that I have only worked on for a
day,
but hopefully you guys agree that it is a good start towards a
consistent
feedback mechanism for the community to take advantage of the
different
distributed CI installations.

Please voice your feedback and concerns.  I am sure I have not
thought
of
everything and we may still want to make fundamental changes to the
approach, but I think the concept is solid.

Cheers,

Will




--
Daan






--
Daan





DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite,
a Persistent Systems business. It is intended only for the use of the individual or entity
to which it is addressed. If you are not the intended recipient, you are not authorized to
read, retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent
Systems business does not accept any liability for virus infected mails.




DISCLAIMER
==========
This e-mail may contain privileged and confidential information which is the property of Accelerite,
a Persistent Systems business. It is intended only for the use of the individual or entity
to which it is addressed. If you are not the intended recipient, you are not authorized to
read, retain, copy, print, distribute or use this message. If you have received this communication
in error, please notify the sender and delete all copies of this message. Accelerite, a Persistent
Systems business does not accept any liability for virus infected mails.
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message