cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hugo Trippaers <h...@trippaers.nl>
Subject Re: Integration Tests on TravisCI
Date Thu, 21 Aug 2014 12:10:08 GMT
Hey guys,

Just to make sure we are all on the same page on this.

There are several initiatives to bolster our CI capabilities and some of them are overlapping.

* There are ongoing discussion to push for a CI environment at the ASF
* There are a number of companies donating hardware/cloud instances to beef up jenkins.buildacloud.org
* Edison is doing CI builds on Amazon i believe
* There is this TravisCI initiative

While they are all very good, overlapping effort means we are wasting somebodies time somewhere.
With a delicate a subject as CI we should try to avoid that i think.

I’ll happily stop working on the CI stuff in jenkins.buildacloud.org for example and focus
my efforts on Travis. So i just want to know where we want to put our effort.

Cheers,

Hugo




On 21 aug. 2014, at 14:02, Ian Duffy <ian@ianduffy.ie> wrote:

> Santosh,
> 
> What command/flags did you use for the above test output?
> 
> 
> On 14 August 2014 17:22, Santhosh Edukulla <santhosh.edukulla@citrix.com>
> wrote:
> 
>> Ian,
>> 
>> I just installed latest marvin after cloudstack build on master, and ran
>> tests against the simulator from smoke folder, below are the results. All
>> passed, except one, i believe that as well could be some global param or
>> some config issue. I hope you want to install, use latest marvin and test
>> suites from master.
>> 
>> === TestName: test_DeployVmAntiAffinityGroup | Status : SUCCESS ===
>> 
>> === TestName: test_deployvm_firstfit | Status : SUCCESS ===
>> 
>> === TestName: test_deployvm_userconcentrated | Status : SUCCESS ===
>> 
>> === TestName: test_deployvm_userdispersing | Status : SUCCESS ===
>> 
>> === TestName: test_01_create_disk_offering | Status : SUCCESS ===
>> 
>> === TestName: test_02_create_sparse_type_disk_offering | Status : SUCCESS
>> ===
>> 
>> === TestName: test_04_create_fat_type_disk_offering | Status : SUCCESS ===
>> 
>> === TestName: test_02_edit_disk_offering | Status : SUCCESS ===
>> 
>> === TestName: test_03_delete_disk_offering | Status : SUCCESS ===
>> 
>> === TestName: test_UpdateConfigParamWithScope | Status : SUCCESS ===
>> 
>> === TestName: test_dedicateGuestVlanRange | Status : SUCCESS ===
>> 
>> === TestName: test_01_create_iso | Status : SUCCESS ===
>> 
>> === TestName: test_02_edit_iso | Status : SUCCESS ===
>> 
>> === TestName: test_03_delete_iso | Status : SUCCESS ===
>> 
>> === TestName: test_05_iso_permissions | Status : SUCCESS ===
>> 
>> === TestName: test_nic_secondaryip_add_remove | Status : SUCCESS ===
>> 
>> === TestName: test_delete_account | Status : SUCCESS ===
>> 
>> === TestName: test_public_ip_admin_account | Status : SUCCESS ===
>> 
>> === TestName: test_public_ip_user_account | Status : SUCCESS ===
>> 
>> === TestName: test_releaseIP | Status : SUCCESS ===
>> 
>> === TestName: test_extendPhysicalNetworkVlan | Status : SUCCESS ===
>> 
>> === TestName: test_UpdateStorageOverProvisioningFactor | Status : SUCCESS
>> ===
>> 
>> === TestName: test_createPortablePublicIPAcquire | Status : SUCCESS ===
>> 
>> === TestName: test_createPortablePublicIPRange | Status : SUCCESS ===
>> 
>> === TestName: test_01_primary_storage_nfs | Status : SUCCESS ===
>> 
>> === TestName: test_privategw_acl | Status : SUCCESS ===
>> 
>> === TestName: test_dedicatePublicIpRange | Status : SUCCESS ===
>> 
>> === TestName: test_create_pvlan_network | Status : SUCCESS ===
>> 
>> === TestName: test_createRegion | Status : SUCCESS ===
>> 
>> === TestName: test_01_reset_vm_on_reboot | Status : SUCCESS ===
>> 
>> === TestName: test_01_updatevolumedetail | Status : SUCCESS ===
>> 
>> === TestName: test_03_restart_network_cleanup | Status : SUCCESS ===
>> 
>> === TestName: test_05_router_basic | Status : SUCCESS ===
>> 
>> === TestName: test_06_router_advanced | Status : SUCCESS ===
>> 
>> === TestName: test_07_stop_router | Status : SUCCESS ===
>> 
>> === TestName: test_08_start_router | Status : SUCCESS ===
>> 
>> === TestName: test_09_reboot_router | Status : SUCCESS ===
>> 
>> === TestName: test_01_sys_vm_start | Status : SUCCESS ===
>> 
>> === TestName: test_02_sys_template_ready | Status : SUCCESS ===
>> 
>> === TestName: test_01_create_service_offering | Status : SUCCESS ===
>> 
>> === TestName: test_02_edit_service_offering | Status : SUCCESS ===
>> 
>> === TestName: test_03_delete_service_offering | Status : SUCCESS ===
>> 
>> === TestName: test_01_list_sec_storage_vm | Status : SUCCESS ===
>> 
>> === TestName: test_02_list_cpvm_vm | Status : SUCCESS ===
>> 
>> === TestName: test_01_create_template | Status : SUCCESS ===
>> 
>> === TestName: test_02_edit_template | Status : SUCCESS ===
>> 
>> === TestName: test_03_delete_template | Status : SUCCESS ===
>> 
>> === TestName: test_05_template_permissions | Status : SUCCESS ===
>> 
>> === TestName: test_07_list_public_templates | Status : SUCCESS ===
>> 
>> === TestName: test_08_list_system_templates | Status : SUCCESS ===
>> 
>> === TestName: test_advZoneVirtualRouter | Status : SUCCESS ===
>> 
>> === TestName: test_deploy_vm | Status : SUCCESS ===
>> 
>> === TestName: test_deploy_vm_multiple | Status : SUCCESS ===
>> 
>> === TestName: test_01_stop_vm | Status : SUCCESS ===
>> 
>> === TestName: test_02_start_vm | Status : SUCCESS ===
>> 
>> === TestName: test_03_reboot_vm | Status : SUCCESS ===
>> 
>> === TestName: test_06_destroy_vm | Status : SUCCESS ===
>> 
>> === TestName: test_07_restore_vm | Status : SUCCESS ===
>> 
>> === TestName: test_08_migrate_vm | Status : SUCCESS ===
>> 
>> === TestName: test_09_expunge_vm | Status : SUCCESS ===
>> 
>> === TestName: test_03_download_attached_volume | Status : SUCCESS ===
>> 
>> === TestName: test_04_delete_attached_volume | Status : SUCCESS ===
>> 
>> === TestName: test_05_detach_volume | Status : SUCCESS ===
>> 
>> === TestName: test_09_delete_detached_volume | Status : SUCCESS ===
>> 
>> === TestName: test_vpc_remote_access_vpn | Status : SUCCESS ===
>> 
>> === TestName: test_vpc_site2site_vpn | Status : SUCCESS ===
>> 
>> 
>> Thanks!
>> Santhosh
>> ________________________________________
>> From: Santhosh Edukulla
>> Sent: Thursday, August 14, 2014 11:23 AM
>> To: dev@cloudstack.apache.org
>> Subject: RE: Integration Tests on TravisCI
>> 
>> Few test suites under smoke/misc folder say test_vm_ha,test_vm_sync,
>> test_deploy_vm have a dependency to run only on simulator and sequentially
>> with no entries available in CS, these are written using simulator mocks,
>> so if you run along with other test suites they will fail.
>> 
>> with -w switch, nose will run all along with other suites under smoke, so,
>> you can use noseexclude plugin and exclude them from normal run, and later
>> run them in serially with out any dependencies. To exclude add the switch
>> --exclude-dir="../smoke/misc" to nose command.
>> 
>> Other should work. Where can we see the detailed logs for these runs?
>> 
>> Regards,
>> Santhosh
>> ________________________________________
>> From: Ian Duffy [ian@ianduffy.ie]
>> Sent: Thursday, August 14, 2014 9:42 AM
>> To: CloudStack Dev
>> Subject: Integration Tests on TravisCI
>> 
>> Hi All,
>> 
>> TL;DR: See https://github.com/imduffy15/Cloudstack-Clone specifically
>> .travis.yml and tools/travis/*
>> 
>> Myself, Sebastien and Rohit have been looking at executing Cloudstack's
>> integration tests against the simulator using http://travis-ci.org.
>> 
>> Doing this will enable us to execute the integration tests on all branches
>> for free. It also means all community forks could make use of the travis-ci
>> service too.
>> 
>> 
>> Currently all tests are executing, there appears to be some tests failing
>> on the simulator specifically:
>> 
>> test_routers test_secondary_storage test_service_offerings test_ssvm
>> test_templates test_vm_ha test_vm_life_cycle test_vm_sync test_volumes
>> test_vpc_vpn
>> 
>> Running these tests both locally and on travis-ci on the 4.4 branch I'm
>> seeing failures. Anybody know why?
>> 
>> Full console output of test execution available at
>> https://travis-ci.org/imduffy15/Cloudstack-Clone/builds/32530691
>> 
>> For those interested in how all this is working:
>> 
>> I have a fork of the Cloudstack source on my github:
>> https://github.com/imduffy15/Cloudstack-Clone
>> 
>> In the root of this repository you will find a .travis.yml file. This file
>> executes a few different stages:
>> 
>> before install: Installs all dependencies and build tools.
>> install: compiles cloudstack
>> before_script: starts up cloudstack and deploys the setup/dev/advanced.cfg
>> script: executes the nosetests
>> 
>> One limitation with travis-ci is that a job is not allowed to run for
>> longer than 50mins.
>> 
>> In order to get around this 3 separate environments are provisioned. They
>> are described as below:
>> 
>> Environment 1 runs test_affinity_groups test_deploy_vm
>> test_deploy_vms_with_varied_deploymentplanners
>>  test_disk_offerings test_global_settings test_guest_vlan_range test_iso
>> test_multipleips_per_nic
>>  test_network test_non_contigiousvlan
>> 
>> Environment 2 runs: test_over_provisioning test_portable_publicip
>> test_primary_storage test_privategw_acl
>>  test_public_ip_range test_pvlan test_regions test_reset_vm_on_reboot
>> test_resource_detail"
>> 
>> Environment 3 runs: test_routers test_secondary_storage
>> test_service_offerings test_ssvm test_templates
>>  test_vm_ha test_vm_life_cycle test_vm_sync test_volumes test_vpc_vpn
>> 
>> Thanks!
>> 
>> Ian
>> 


Mime
View raw message