cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ian Duffy <...@ianduffy.ie>
Subject Re: How to speed up testing using BVT/smoke tests with Simulator
Date Mon, 04 Aug 2014 19:31:34 GMT
Rohit,

Those changes cause conflicts when attempting to deploy the devcloud.sql
file as they specify the same primary keys.


On 1 August 2014 15:56, Rohit Yadav <rohit.yadav@shapeblue.com> wrote:

> I’ve added config in developer/developer-prefill.sql to have three new
> config params commonly used by devs and to speed up account cleanup (60s
> now).
>
> 137655d4e11cb490391d0f4982c52262bbb7427c master
> a68d8986829c462d655bf59a0cbcc0d4e8b1cc3e 4.4
>
> Regards.
>
> On 01-Aug-2014, at 7:27 am, Santhosh Edukulla <
> santhosh.edukulla@citrix.com> wrote:
>
> > 1. yes, test cases currently run in parallel in ci environment, using
> parallel nose commands.
> >
> > 2. There were few  hard codings for sleep, where we removed at some
> places, still there could be few\many out there.
> >
> > 3. 1 sec sleep and poll check is too heavy i believe, for async. So,
> worst a test case can add time of execution by 5 seconds more, compared to
> succesful operation time.
> >
> > Santhosh
> > ________________________________________
> > From: Rohit Yadav [rohit.yadav@shapeblue.com]
> > Sent: Thursday, July 31, 2014 7:15 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: How to speed up testing using BVT/smoke tests with Simulator
> >
> > Hi Edison,
> >
> > Thanks for the pointers! I’ll try them out and see if there is way to do
> it on Travis/CloudBees as well and I hope other people will religiously
> start using simulator/bvt (at least the basic ones) for their check-ins.
> >
> > Regards.
> >
> > On 01-Aug-2014, at 12:15 am, Edison Su <Edison.su@citrix.com> wrote:
> >
> >>
> >>
> >>> -----Original Message-----
> >>> From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
> >>> Sent: Thursday, July 31, 2014 5:12 AM
> >>> To: dev@cloudstack.apache.org
> >>> Subject: How to speed up testing using BVT/smoke tests with Simulator
> >>>
> >>> Hi,
> >>>
> >>> Santosh put together a good wiki page on how to validate local changes
> using
> >>> our Python/marvin based build verification tests (path:
> >>> test/integration/smoke):
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Validating+check
> >>> -ins+for+your+local+changes%2C+using+Simulator
> >>>
> >>> I've a mini PC for this and using CloudStack 4.4.0 to build/test
> CloudStack
> >>> 4.4/master branch on it in a VM. Some of us are also exploring
> free/cheap CI
> >>> services such as Travis, CloudBees etc. which can be used by
> developers to
> >>> test their check-ins. If anyone of you have tried something like this
> please
> >>> share.
> >> Today, I tried to build and test CloudStack on a super powerful machine
> provided by Azure. Imaging, build & test on a 16 Cores, 120G machine, it
> should be awesome, and most importantly, it's FREE. You can get a free MSDN
> subscription fromhttps://
> svn.apache.org/repos/private/committers/donated-licenses/msdn-subscription.html,
> after that, you will get $150 credit monthly in Azure. For build &test
> only, $150 is good enough.
> >>
> >>>
> >>> This is how I build CloudStack for validating with simulator:
> >>>   mvn -Pdeveloper -Dsimulator clean install
> >>>   mvn -Pdeveloper -pl developer -Ddeploydb
> >>>   mvn -Pdeveloper -pl developer -Ddeploydb-simulator
> >>>   mvn -pl client jetty:run -Dsimulator
> >>>
> >>> And finally run smoke tests (BVT):
> >>> nosetests --with-marvin --marvin-config=setup/dev/advanced.cfg --with-
> >>> xunit --xunit-file=/tmp/bvt_selfservice_cases.xml -a
> >>> tags=advanced,required_hardware=false -w test/integration/smoke --
> >>> hypervisor=simulator
> >>>
> >>> It currently took 50 mins on my setup. How can we speed it up, say by
> >>> reducing global variable timeout settings etc? Should we reduce
> timeouts etc.
> >>> in deploydb-simulator specific sql files?
> >>
> >> There several places we can improve the marvin test:
> >> 1. queryAsyncJob waits 5 second for each call, can change to 1s.
> >> 2. There are hardcoded sleep in test code, such base.py, search
> time.sleep
> >> 3. global configuration: account.cleanup.interval sets to 600s, so the
> test suite will stop for 10 minutes after running for a while.
> >> 3. most importantly, if we can run the test cases in parallel, then
> speedup should be great. Does anybody try to run it in parallel before?
> >>
> >>>
> >>> Regards,
> >>> Rohit Yadav
> >>> Software Architect, ShapeBlue
> >>> M. +41 779015219 | rohit.yadav@shapeblue.com
> >>> Blog: bhaisaab.org | Twitter: @_bhaisaab
> >>>
> >>>
> >>>
> >>>
> >>> Find out more about ShapeBlue and our range of CloudStack related
> services
> >>>
> >>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-
> >>> build//>
> >>> CSForge - rapid IaaS deployment
> >>> framework<http://shapeblue.com/csforge/>
> >>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> >>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-
> >>> infrastructure-support/>
> >>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-
> >>> training/>
> >>>
> >>> This email and any attachments to it may be confidential and are
> intended
> >>> solely for the use of the individual to whom it is addressed. Any
> views or
> >>> opinions expressed are solely those of the author and do not
> necessarily
> >>> represent those of Shape Blue Ltd or related companies. If you are not
> the
> >>> intended recipient of this email, you must neither take any action
> based
> >>> upon its contents, nor copy or show it to anyone. Please contact the
> sender if
> >>> you believe you have received this email in error. Shape Blue Ltd is a
> >>> company incorporated in England & Wales. ShapeBlue Services India LLP
> is a
> >>> company incorporated in India and is operated under license from Shape
> >>> Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated
> in
> >>> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA
> Pty
> >>> Ltd is a company registered by The Republic of South Africa and is
> traded
> >>> under license from Shape Blue Ltd. ShapeBlue is a registered trademark.
> >
> > Regards,
> > Rohit Yadav
> > Software Architect, ShapeBlue
> > M. +41 779015219 | rohit.yadav@shapeblue.com
> > Blog: bhaisaab.org | Twitter: @_bhaisaab
> >
> >
> >
> >
> > Find out more about ShapeBlue and our range of CloudStack related
> services
> >
> > IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> > CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> > CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> > CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
> >
> > This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed. Any
> views or opinions expressed are solely those of the author and do not
> necessarily represent those of Shape Blue Ltd or related companies. If you
> are not the intended recipient of this email, you must neither take any
> action based upon its contents, nor copy or show it to anyone. Please
> contact the sender if you believe you have received this email in error.
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> Services India LLP is a company incorporated in India and is operated under
> license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a
> company incorporated in Brasil and is operated under license from Shape
> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of
> South Africa and is traded under license from Shape Blue Ltd. ShapeBlue is
> a registered trademark.
>
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.yadav@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related services
>
> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views or
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not the
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the sender
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is a
> company incorporated in India and is operated under license from Shape Blue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message