Mailing-List: contact dev-help@cloudstack.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cloudstack.apache.org
Received-SPF: pass (nike.apache.org: domain of ian@ianduffy.ie designates
 209.85.217.180 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <44271392-3586-4CB5-BE26-565F249BC44E@shapeblue.com>
References: <3C1A39CD-ECAF-44E5-AECC-F174A2F4ECE4@shapeblue.com>
	<77B337AF224FD84CBF8401947098DD8731217291@SJCPEX01CL01.citrite.net>
	<3D5F7FCF-E656-46BE-AB9B-04CEE4277D97@shapeblue.com>
	<CB688D716959224C95995F52DE5E82424A958F3C@SINPEX01CL02.citrite.net>
	<44271392-3586-4CB5-BE26-565F249BC44E@shapeblue.com>
Date: Mon, 4 Aug 2014 20:31:34 +0100
Message-ID: 
 <CAEJ3w4U5opfnXgkk3xMBDhSqmqa0aCWotD0_XkvY-TdgRat2HA@mail.gmail.com>
Subject: Re: How to speed up testing using BVT/smoke tests with Simulator
From: Ian Duffy <ian@ianduffy.ie>
To: CloudStack Dev <dev@cloudstack.apache.org>
Content-Type: multipart/alternative; boundary=001a11c363ec6b2c4304ffd2c9a0

--001a11c363ec6b2c4304ffd2c9a0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

Rohit,

Those changes cause conflicts when attempting to deploy the devcloud.sql
file as they specify the same primary keys.


On 1 August 2014 15:56, Rohit Yadav <rohit.yadav@shapeblue.com> wrote:

> I=E2=80=99ve added config in developer/developer-prefill.sql to have thre=
e new
> config params commonly used by devs and to speed up account cleanup (60s
> now).
>
> 137655d4e11cb490391d0f4982c52262bbb7427c master
> a68d8986829c462d655bf59a0cbcc0d4e8b1cc3e 4.4
>
> Regards.
>
> On 01-Aug-2014, at 7:27 am, Santhosh Edukulla <
> santhosh.edukulla@citrix.com> wrote:
>
> > 1. yes, test cases currently run in parallel in ci environment, using
> parallel nose commands.
> >
> > 2. There were few  hard codings for sleep, where we removed at some
> places, still there could be few\many out there.
> >
> > 3. 1 sec sleep and poll check is too heavy i believe, for async. So,
> worst a test case can add time of execution by 5 seconds more, compared t=
o
> succesful operation time.
> >
> > Santhosh
> > ________________________________________
> > From: Rohit Yadav [rohit.yadav@shapeblue.com]
> > Sent: Thursday, July 31, 2014 7:15 PM
> > To: dev@cloudstack.apache.org
> > Subject: Re: How to speed up testing using BVT/smoke tests with Simulat=
or
> >
> > Hi Edison,
> >
> > Thanks for the pointers! I=E2=80=99ll try them out and see if there is =
way to do
> it on Travis/CloudBees as well and I hope other people will religiously
> start using simulator/bvt (at least the basic ones) for their check-ins.
> >
> > Regards.
> >
> > On 01-Aug-2014, at 12:15 am, Edison Su <Edison.su@citrix.com> wrote:
> >
> >>
> >>
> >>> -----Original Message-----
> >>> From: Rohit Yadav [mailto:rohit.yadav@shapeblue.com]
> >>> Sent: Thursday, July 31, 2014 5:12 AM
> >>> To: dev@cloudstack.apache.org
> >>> Subject: How to speed up testing using BVT/smoke tests with Simulator
> >>>
> >>> Hi,
> >>>
> >>> Santosh put together a good wiki page on how to validate local change=
s
> using
> >>> our Python/marvin based build verification tests (path:
> >>> test/integration/smoke):
> >>>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Validating+check
> >>> -ins+for+your+local+changes%2C+using+Simulator
> >>>
> >>> I've a mini PC for this and using CloudStack 4.4.0 to build/test
> CloudStack
> >>> 4.4/master branch on it in a VM. Some of us are also exploring
> free/cheap CI
> >>> services such as Travis, CloudBees etc. which can be used by
> developers to
> >>> test their check-ins. If anyone of you have tried something like this
> please
> >>> share.
> >> Today, I tried to build and test CloudStack on a super powerful machin=
e
> provided by Azure. Imaging, build & test on a 16 Cores, 120G machine, it
> should be awesome, and most importantly, it's FREE. You can get a free MS=
DN
> subscription fromhttps://
> svn.apache.org/repos/private/committers/donated-licenses/msdn-subscriptio=
n.html,
> after that, you will get $150 credit monthly in Azure. For build &test
> only, $150 is good enough.
> >>
> >>>
> >>> This is how I build CloudStack for validating with simulator:
> >>>   mvn -Pdeveloper -Dsimulator clean install
> >>>   mvn -Pdeveloper -pl developer -Ddeploydb
> >>>   mvn -Pdeveloper -pl developer -Ddeploydb-simulator
> >>>   mvn -pl client jetty:run -Dsimulator
> >>>
> >>> And finally run smoke tests (BVT):
> >>> nosetests --with-marvin --marvin-config=3Dsetup/dev/advanced.cfg --wi=
th-
> >>> xunit --xunit-file=3D/tmp/bvt_selfservice_cases.xml -a
> >>> tags=3Dadvanced,required_hardware=3Dfalse -w test/integration/smoke -=
-
> >>> hypervisor=3Dsimulator
> >>>
> >>> It currently took 50 mins on my setup. How can we speed it up, say by
> >>> reducing global variable timeout settings etc? Should we reduce
> timeouts etc.
> >>> in deploydb-simulator specific sql files?
> >>
> >> There several places we can improve the marvin test:
> >> 1. queryAsyncJob waits 5 second for each call, can change to 1s.
> >> 2. There are hardcoded sleep in test code, such base.py, search
> time.sleep
> >> 3. global configuration: account.cleanup.interval sets to 600s, so the
> test suite will stop for 10 minutes after running for a while.
> >> 3. most importantly, if we can run the test cases in parallel, then
> speedup should be great. Does anybody try to run it in parallel before?
> >>
> >>>
> >>> Regards,
> >>> Rohit Yadav
> >>> Software Architect, ShapeBlue
> >>> M. +41 779015219 | rohit.yadav@shapeblue.com
> >>> Blog: bhaisaab.org | Twitter: @_bhaisaab
> >>>
> >>>
> >>>
> >>>
> >>> Find out more about ShapeBlue and our range of CloudStack related
> services
> >>>
> >>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-
> >>> build//>
> >>> CSForge - rapid IaaS deployment
> >>> framework<http://shapeblue.com/csforge/>
> >>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> >>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-
> >>> infrastructure-support/>
> >>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-
> >>> training/>
> >>>
> >>> This email and any attachments to it may be confidential and are
> intended
> >>> solely for the use of the individual to whom it is addressed. Any
> views or
> >>> opinions expressed are solely those of the author and do not
> necessarily
> >>> represent those of Shape Blue Ltd or related companies. If you are no=
t
> the
> >>> intended recipient of this email, you must neither take any action
> based
> >>> upon its contents, nor copy or show it to anyone. Please contact the
> sender if
> >>> you believe you have received this email in error. Shape Blue Ltd is =
a
> >>> company incorporated in England & Wales. ShapeBlue Services India LLP
> is a
> >>> company incorporated in India and is operated under license from Shap=
e
> >>> Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporate=
d
> in
> >>> Brasil and is operated under license from Shape Blue Ltd. ShapeBlue S=
A
> Pty
> >>> Ltd is a company registered by The Republic of South Africa and is
> traded
> >>> under license from Shape Blue Ltd. ShapeBlue is a registered trademar=
k.
> >
> > Regards,
> > Rohit Yadav
> > Software Architect, ShapeBlue
> > M. +41 779015219 | rohit.yadav@shapeblue.com
> > Blog: bhaisaab.org | Twitter: @_bhaisaab
> >
> >
> >
> >
> > Find out more about ShapeBlue and our range of CloudStack related
> services
> >
> > IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> > CSForge =E2=80=93 rapid IaaS deployment framework<http://shapeblue.com/=
csforge/>
> > CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> > CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> > CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
> >
> > This email and any attachments to it may be confidential and are
> intended solely for the use of the individual to whom it is addressed. An=
y
> views or opinions expressed are solely those of the author and do not
> necessarily represent those of Shape Blue Ltd or related companies. If yo=
u
> are not the intended recipient of this email, you must neither take any
> action based upon its contents, nor copy or show it to anyone. Please
> contact the sender if you believe you have received this email in error.
> Shape Blue Ltd is a company incorporated in England & Wales. ShapeBlue
> Services India LLP is a company incorporated in India and is operated und=
er
> license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a
> company incorporated in Brasil and is operated under license from Shape
> Blue Ltd. ShapeBlue SA Pty Ltd is a company registered by The Republic of
> South Africa and is traded under license from Shape Blue Ltd. ShapeBlue i=
s
> a registered trademark.
>
> Regards,
> Rohit Yadav
> Software Architect, ShapeBlue
> M. +41 779015219 | rohit.yadav@shapeblue.com
> Blog: bhaisaab.org | Twitter: @_bhaisaab
>
>
>
>
> Find out more about ShapeBlue and our range of CloudStack related service=
s
>
> IaaS Cloud Design & Build<
> http://shapeblue.com/iaas-cloud-design-and-build//>
> CSForge =E2=80=93 rapid IaaS deployment framework<http://shapeblue.com/cs=
forge/>
> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
> CloudStack Infrastructure Support<
> http://shapeblue.com/cloudstack-infrastructure-support/>
> CloudStack Bootcamp Training Courses<
> http://shapeblue.com/cloudstack-training/>
>
> This email and any attachments to it may be confidential and are intended
> solely for the use of the individual to whom it is addressed. Any views o=
r
> opinions expressed are solely those of the author and do not necessarily
> represent those of Shape Blue Ltd or related companies. If you are not th=
e
> intended recipient of this email, you must neither take any action based
> upon its contents, nor copy or show it to anyone. Please contact the send=
er
> if you believe you have received this email in error. Shape Blue Ltd is a
> company incorporated in England & Wales. ShapeBlue Services India LLP is =
a
> company incorporated in India and is operated under license from Shape Bl=
ue
> Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Bras=
il
> and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd i=
s
> a company registered by The Republic of South Africa and is traded under
> license from Shape Blue Ltd. ShapeBlue is a registered trademark.
>

--001a11c363ec6b2c4304ffd2c9a0--