airavata-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Shenoy, Gourav Ganesh" <goshe...@indiana.edu>
Subject Re: List of Devops Scripts for Apache Airavata
Date Fri, 10 Mar 2017 18:14:00 GMT
Hi Marcus,

Although it is not being actively used, but it still is in working shape.

Thanks and Regards,
Gourav Shenoy

From: "Christie, Marcus Aaron" <machrist@iu.edu>
Date: Friday, March 10, 2017 at 11:54 AM
To: "dev@airavata.apache.org" <dev@airavata.apache.org>, "Shenoy, Gourav Ganesh" <goshenoy@indiana.edu>
Subject: Re: List of Devops Scripts for Apache Airavata

Dev,

Eric and I met this morning to discuss how to consolidate these various efforts.  Here’s
what we came up with:

1. (Marcus) Create an airavata-devops repo for holding all of the Ansible scripts.  To start
with, we’ll use the dev-tools/ansible scripts.
2. (Eric) Once the airavata-devops repo is in place, Eric will move his Jetstream provisioning
scripts onto a branch there
3. (Eric) Eric is also going to start working on Jetstream scripts for other servers besides
PGA

A question for Gourav: are the modules/cloud/ansible-playbooks still being actively used?

Some other things we discussed:
* How to securely store secret information?  We discussed using ansible-vault versus having
a separate, private repo for Ansible inventories.  We settled on using ansible-vault since
it has the advantage of keeping everything in one repo. The disadvantage is in needing to
share a password file out-of-band with the rest of the team.  However, we felt the sharing
of the password would be mitigated as we move toward more automated systems for running deployments
(for example, users who only need to run a deploy won’t need the password file if we use
Jenkins to run deploys; only Jenkins would really need to have the password file).
* One advantage to NOT having a separate repo for devops scripts is that it keeps the devops
scripts versioned together with the code those scripts can build and deploy.  Having a separate
repo means we will need to have some discipline when updating devops scripts.  I think it
makes sense to have the devops repo branch structure reflect the airavata branch structure.
 That is, the ‘develop’ branch of the devops repo should be good for deploying the ‘develop’
branch of airavata and likewise for ‘master’.  If code is updated in airavata on a separate
branch that needs corresponding devops script updates, that should likewise be done on a same-named
separate branch in the devops repo.

Eric, let me know if I missed anything.


Thanks,

Marcus


On Mar 9, 2017, at 3:37 PM, Anuj Bhandar <bhandar.anuj@gmail.com<mailto:bhandar.anuj@gmail.com>>
wrote:

Gourav,

It is a great initiative, I vote for a separate repository for devops, the Airavata repository
plays many roles already.

A separate repository would aid active development and bring modularity.

Thanks,
Anuj Bhandar

On 3/9/17 3:14 PM, Coulter, John Eric wrote:
Thanks for starting this, Gourav!
I'm inclined to vote for a separate DevOps repo, to keep things more modular.

I know Marcus and I have done some work/testing starting from scripts that I think Shameera
created in dev-tools/ansible. I've got a side-repo up which contains playbooks for provisioning
and deploying the PGA from scratch as a proof of concept, using dynamic inventory instead
of a static file (so, on an empty jetstream allocation, you get working instance running the
PGA, with router, public ip with a single ansible-playbook command). Next step there is to
add provisioning for VMs to run airavata, etc.

Script location

Branch

Purpose of Script

modules/cloud/ansible-playbooks​​​

develop

1. Provisions instances on EC2 and OpenStack (Jetstream)
2. Deploys a Mesos/Marathon cluster on the provisioned i​nstances
 ​

​dev-tools/ansible/

​develop

​1. Deploy/update airavata services on existing machines​/instances
2. todo - include provisioning scripts for cloud resources

https://github.com/ECoulter/airavata-vms

​

​1. POC for provisioning and deployment of PGA on Jetstream, with dynamic inventory. (intended
to merge with dev-tools/ansible when complete/tested by others)



---------------

Cheers,
---------------------
Eric Coulter        jecoulte@iu.edu<mailto:jecoulte@iu.edu>
XSEDE Capabilities and Resource Integration Engineer
IU Campus Bridging & Research Infrastructure
RT/PTI/UITS
812-856-3250



Mime
View raw message