provisionr-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrei Savu <savu.and...@gmail.com>
Subject How to build and test a release candidate
Date Thu, 20 Jun 2013 19:40:57 GMT
Hi -

Warning: long email

We are missing a lot of docs right now so I wanted to share with you how
I've built and tested the current release candidate - as some sort of
documentation we can turn into a wiki page.

I've started by making sure that all the existing tests run as expected on
the master branch: unit, integration and live tests for Amazon EC2. The
support for CloudStack is very limited and not useful yet.

Commands:

mvn -Pwith-assembly clean install    # unit & integration

./scripts/activities_test.sh amazon    # individual Activiti activities test
./scripts/process_test.sh amazon     # test the pool management process as
a whole

I takes 15-20 or more to execute everything and you need to have your cloud
providers in ~/.m2/settings.xml as documented here:

https://cwiki.apache.org/confluence/display/PROVISIONR/How+To+Release

Next step was to deploy the Maven artifacts to Nexus staging and update the
version numbers on the way using the following simple command:

./scripts/release.sh 0.4.0-incubating 0.5.0-incubating-SNAPSHOT

This also takes a while to complete. When done I've closed the staging
repository at repository.apache.org and pushed the new master branch and
the tag.

As the last step I've signed and checksummed
provisionr-0.4.0-incubating[-src].tar.gz and uploaded everything to
people.apache.org in my home directory.

----

Now let's check the release candidate. I will start by downloading the
files from people.apache.org in a temporary folder.

$ wget -r -l1 --no-parent -A "*.tar.gz*"
http://people.apache.org/~asavu/provisionr-0.4.0-incubating-candidate-0/ -nd

Verify the checksums:

$ shasum -c provisionr-0.4.0-incubating-src.tar.gz.sha1
provisionr-0.4.0-incubating-src.tar.gz: OK

$ shasum -c provisionr-0.4.0-incubating.tar.gz.sha1
provisionr-0.4.0-incubating.tar.gz: OK

$ cat *.md5
MD5 (provisionr-0.4.0-incubating-src.tar.gz) =
2c7af6a5726feec59c7ee93698e591ed
MD5 (provisionr-0.4.0-incubating.tar.gz) = 77ce3dc0b2f18804d14d15f7204070c1

$ md5sum *.tar.gz
2c7af6a5726feec59c7ee93698e591ed  provisionr-0.4.0-incubating-src.tar.gz
77ce3dc0b2f18804d14d15f7204070c1  provisionr-0.4.0-incubating.tar.gz

Verify the signatures:

$ wget https://dist.apache.org/repos/dist/release/incubator/provisionr/KEYS
$ gpg --import KEYS
gpg: key ACCD51A7: "Andrei Savu (CODE SIGNING KEY) <asavu@apache.org>" not
changed
gpg: Total number processed: 1
gpg:              unchanged: 1

$ gpg --verify provisionr-0.4.0-incubating-src.tar.gz.asc
gpg: Signature made Thu 20 Jun 14:36:22 2013 EEST using RSA key ID ACCD51A7
gpg: Good signature from "Andrei Savu (CODE SIGNING KEY) <asavu@apache.org>"

$ gpg --verify provisionr-0.4.0-incubating.tar.gz.asc
gpg: Signature made Thu 20 Jun 14:37:38 2013 EEST using RSA key ID ACCD51A7
gpg: Good signature from "Andrei Savu (CODE SIGNING KEY) <asavu@apache.org>"

----

I will assume the source release is OK if we can re-build everything with
an empty local Maven repository:

$ rm -rf ~/.m2/repository/*
$ tar xvfz provisionr-0.4.0-incubating-src.tar.gz
$ cd provisionr-0.4.0-incubating-src

$ mvn -Pwith-assembly clean install
[...]
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 15:20.286s
[...]

Maven also runs the Rat check for each module.

----

Now let's test the binary release - a custom Karaf distribution that
contains everything you need to start an Apache Provisionr instance on the
local machine:

$ tar xvfz provisionr-0.4.0-incubating.tar.gz
$ cd provisionr-0.4.0-incubating
$ ./bin/provisionr
# .. and now you should see an interactive shell

Use the following command to check for exceptions in the log file:

Apache Provisionr [0.4.0-incubating] $ log:display-exception
# you should get no results

Other interesting commands:

Apache Provisionr [0.4.0-incubating] $ provisionr:services
Services: amazon
Apache Provisionr [0.4.0-incubating] $ provisionr:templates
- jenkins
A short template that installs the latest jenkins with git & svn
- cdh4
Cloudera CDH4 template for Ubuntu 10.04 LTS (lucid)
- cdh3
Cloudera CDH3 template for Ubuntu 10.04 LTS (lucid) including the Cloudera
Manager installer
Apache Provisionr [0.4.0-incubating] $ provisionr:pools
No active pools found. You can create one using provisionr:create

Also we can check how Activiti is doing:

Apache Provisionr [0.4.0-incubating] $ activiti:list

Activiti Deployments
--------------------
 ID   Name                          Deployment Time
[1  ][org.apache.provisionr.amazon][Jun 20, 2013 9:55:44 PM]

Activiti Process Definitions
----------------------------
 Definition ID             Name                            Ver  Resource
[amazonPoolManagement:1:6][Amazon Pool Management Process][1
 ][OSGI-INF/activiti/amazonPoolManagement.bpmn20.xml]
[amazonMachineSetup:1:7  ][Amazon Machine Setup          ][1
 ][OSGI-INF/activiti/amazonMachineSetup.bpmn20.xml  ]

History of Activiti Process Instances
-------------------------------------
No History on Activiti Processes.

Active Process Instances
------------------------
No Active Process Instances Found.

The Activiti Explorer is available at the following URL:

http://localhost:8181/activiti-explorer/
Login: kermit:kermit

Check the "Processes" tab for some nice looking diagrams.

Before creating a pool you have to add your credentials to:
$ vim etc/org.apache.provisionr.amazon.cfg

The Amazon provider bundle should be automatically reloaded when this file
is updated.

Let's start two t1.micro's and install Jenkins:

Apache Provisionr [0.4.0-incubating] $ provisionr:create --id amazon -k
test-1 -s 2 -t jenkins --port 8080
Pool management process started (id: 68)

You can check the process status in the Activiti Explorer UI or using the
CLI commands or by watching the log output (log:tail). In ~10 minutes you
should have two machines up and running that have the jenkins package
installed (the server is not running by default).

You can get the IP addresses of the machines by typing:
Apache Provisionr [0.4.0-incubating] $ pools

I've used ssh to login on the remote machine and start the jenkins server.
Provisionr is automatically creating a user with the same name as the local
user and configures key-based authentication using the public key from
~/.ssh/id_rsa.pub:

$ ssh ec2-23-22-125-74.compute-1.amazonaws.com
...@ip-10-154-140-213:~$ sudo service jenkins start
 * Starting Jenkins Continuous Integration Server jenkins
                                     [ OK ]
...@ip-10-154-140-213:~$ sudo apt-cache policy jenkins
jenkins:
  Installed: 1.519
  Candidate: 1.519
  Version table:
 *** 1.519 0
        500 http://pkg.jenkins-ci.org/debian/ binary/ Packages
        100 /var/lib/dpkg/status
     1.424.6+dfsg-1ubuntu0.1 0
        500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/precise-updates/universe
amd64 Packages
        500 http://security.ubuntu.com/ubuntu/ precise-security/universe
amd64 Packages
     1.424.6+dfsg-1 0
        500 http://us-east-1.ec2.archive.ubuntu.com/ubuntu/precise/universe
amd64 Packages

Other interesting URLs:

http://localhost:8181/system/console/bundles   (login karaf:karaf)
http://localhost:8181/rundeck/machines.xml     (custom endpoint for Rundeck
integration)

**** And don't forget to destroy the pool:

Apache Provisionr [0.4.0-incubating] $ provisionr:destroy -k test-1

After a while you should see no running Activiti processes:

Apache Provisionr [0.4.0-incubating] $ activiti:list

Activiti Deployments
--------------------
 ID   Name                          Deployment Time
[1  ][org.apache.provisionr.amazon][Jun 20, 2013 9:55:44 PM]

Activiti Process Definitions
----------------------------
 Definition ID             Name                            Ver  Resource
[amazonPoolManagement:1:6][Amazon Pool Management Process][1
 ][OSGI-INF/activiti/amazonPoolManagement.bpmn20.xml]
[amazonMachineSetup:1:7  ][Amazon Machine Setup          ][1
 ][OSGI-INF/activiti/amazonMachineSetup.bpmn20.xml  ]

History of Activiti Process Instances
-------------------------------------
 Definition ID             Ins  Start Time                End Time
[amazonMachineSetup:1:7  ][128][Jun 20, 2013 10:05:48 PM][Jun 20, 2013
10:08:28 PM]
[amazonMachineSetup:1:7  ][119][Jun 20, 2013 10:05:48 PM][Jun 20, 2013
10:09:06 PM]
[amazonMachineSetup:1:7  ][242][Jun 20, 2013 10:06:50 PM][Jun 20, 2013
10:09:22 PM]
[amazonMachineSetup:1:7  ][251][Jun 20, 2013 10:06:50 PM][Jun 20, 2013
10:10:03 PM]
[amazonPoolManagement:1:6][17 ][Jun 20, 2013 10:04:53 PM][Jun 20, 2013
10:13:24 PM]
[amazonPoolManagement:1:6][68 ][Jun 20, 2013 10:05:42 PM][Jun 20, 2013
10:25:42 PM]

Active Process Instances
------------------------
No Active Process Instances Found.

-----

And the last thing we need to test is the Maven staging repository hosted
at:
https://repository.apache.org/content/repositories/orgapacheprovisionr-035/

I will start from a vanilla Apache Karaf 2.3.1 install downloaded from:
http://karaf.apache.org/index/community/download.html

I've added a few more repositories to etc/org.ops4j.pax.url.mvn.cfg:

org.ops4j.pax.url.mvn.repositories= \
    http://repo1.maven.org/maven2@id=central, \

https://oss.sonatype.org/content/repositories/snapshots@snapshots@noreleases@id=sonatype-snapshot,
\

https://oss.sonatype.org/content/repositories/ops4j-snapshots@snapshots@noreleases@id=ops4j-snapshot,
\

http://repository.apache.org/content/groups/snapshots-group@snapshots@noreleases@id=apache,
\
    http://svn.apache.org/repos/asf/servicemix/m2-repo@id=servicemix, \
    http://repository.springsource.com/maven/bundles/release@id=springsource,
\

http://repository.springsource.com/maven/bundles/external@id=springsourceext,
\
    http://oss.sonatype.org/content/repositories/releases/@id=sonatype, \

https://repository.apache.org/content/repositories/orgapacheprovisionr-035/,
\
    https://maven.alfresco.com/nexus/content/repositories/activiti/, \
    http://maven.thebuzzmedia.com

$ ./bin/karaf
karaf@root> features:addurl
mvn:org.apache.provisionr/provisionr-features/0.4.0-incubating/xml/features
karaf@root> features:install provisionr-all

... and that's all - Apache Provisionr is now up and running in a vanilla
Karaf install. Check the relevant URLs:

http://localhost:8181/activiti-explorer/
http://localhost:8181/rundeck/machines.xml

You can also install the Karaf webconsole if you want:

karaf@root> features:install webconsole
http://localhost:8181/system/console/bundles

IMO we are ready to make this release public unless we still have
unresolved licensing issues.

Regards,

-- Andrei Savu / axemblr.com

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message