cloudstack-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhinandan Prateek <abhinandan.prat...@shapeblue.com>
Subject Re: [marvin tests] interpreting results
Date Fri, 24 Apr 2015 15:14:25 GMT
Hi Gaurav,

  Your réponse will definitely help us in focusing in  the right direction.

Thanks a lot !
-abhi



> On 24-Apr-2015, at 7:11 pm, Gaurav Aradhye <gaurav.aradhye@clogeny.com> wrote:
>
> Abhi, went through the logs. The tests which have FAILED have failed because product
behavior is not as expected in the test.
> Tests with Exception seem have issue in test code.
>
> Regards,
> Gaurav Aradhye
>
> On Apr 24, 2015, at 4:47 PM, Abhinandan Prateek <abhinandan.prateek@shapeblue.com>
wrote:
>
>> Attaching the logs. This is with latest 4.5 branch.
>>
>>
>> Find out more about ShapeBlue and our range of CloudStack related services
>>
>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>> CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
>>
>> This email and any attachments to it may be confidential and are intended solely
for the use of the individual to whom it is addressed. Any views or opinions expressed are
solely those of the author and do not necessarily represent those of Shape Blue Ltd or related
companies. If you are not the intended recipient of this email, you must neither take any
action based upon its contents, nor copy or show it to anyone. Please contact the sender if
you believe you have received this email in error. Shape Blue Ltd is a company incorporated
in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and
is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty
Ltd is a company registered by The Republic of South Africa and is traded under license from
Shape Blue Ltd. ShapeBlue is a registered trademark.
>> <test_acl_isolatednetwork.tar>
>>
>> Regards,
>> -abhi
>>
>>
>>> On 24-Apr-2015, at 4:21 pm, Gaurav Aradhye <gaurav.aradhye@clogeny.com>
wrote:
>>>
>>> Hi Abhi,
>>>
>>> FAILED indicates it’s an assertion error. Even if the test case is expecting
a failure, it must have been written in such a way that when an operation fails, test case
passes.
>>> Here it indicates that operation which should have failed has succeeded, might
be a product issue.
>>>
>>> Exception indicates there is an unexpected exception in the code, other than
assertions.
>>>
>>> Can you please share the detailed logs from failed_plus_exceptions.txt file in
the log folder?
>>>
>>> Regards,
>>> Gaurav Aradhye
>>>
>>> On Apr 24, 2015, at 4:04 PM, Abhinandan Prateek <abhinandan.prateek@shapeblue.com>
wrote:
>>>
>>>>
>>>> Have been trying to run some marvin test cases. Need to know how to interpret
the result.
>>>>
>>>> On running test cases I see several success and failure messages. Lets take
example of each from output of
>>>> /root/cloudstack/test/integration/component/test_acl_isolatednetwork.py on
ACS 4.5, taking a sample from output:
>>>>
>>>> === TestName: test_02_createNetwork_admin_foruserinsamedomain | Status :
SUCCESS ===
>>>>
>>>> ***The above says the test case is successful, basically the functionality
works.
>>>>
>>>> === TestName: test_09_createNetwork_user_foruserinsamedomain | Status : FAILED
===
>>>>
>>>> ***From the code it appears that the test is expected to fail, this also
means the test case is successful.
>>>>
>>>> === TestName: test_12_deployvm_admin_foruserinsamedomain | Status : EXCEPTION
===
>>>>
>>>> ****I assume this means the test case did not execute successfully ? If it
is on a fresh environment will that mean something is wrong with the test itself ?
>>>>
>>>> Can someone share some gyan on above ?
>>>>
>>>> -abhi
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> Find out more about ShapeBlue and our range of CloudStack related services
>>>>
>>>> IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
>>>> CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
>>>> CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
>>>> CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
>>>> CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
>>>> CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>
>>>>
>>>> This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or opinions expressed
are solely those of the author and do not necessarily represent those of Shape Blue Ltd or
related companies. If you are not the intended recipient of this email, you must neither take
any action based upon its contents, nor copy or show it to anyone. Please contact the sender
if you believe you have received this email in error. Shape Blue Ltd is a company incorporated
in England & Wales. ShapeBlue Services India LLP is a company incorporated in India and
is operated under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty
Ltd is a company registered by The Republic of South Africa and is traded under license from
Shape Blue Ltd. ShapeBlue is a registered trademark.
>>>
>>
>

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design & Build<http://shapeblue.com/iaas-cloud-design-and-build//>
CSForge – rapid IaaS deployment framework<http://shapeblue.com/csforge/>
CloudStack Consulting<http://shapeblue.com/cloudstack-consultancy/>
CloudStack Software Engineering<http://shapeblue.com/cloudstack-software-engineering/>
CloudStack Infrastructure Support<http://shapeblue.com/cloudstack-infrastructure-support/>
CloudStack Bootcamp Training Courses<http://shapeblue.com/cloudstack-training/>

This email and any attachments to it may be confidential and are intended solely for the use
of the individual to whom it is addressed. Any views or opinions expressed are solely those
of the author and do not necessarily represent those of Shape Blue Ltd or related companies.
If you are not the intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the sender if you believe
you have received this email in error. Shape Blue Ltd is a company incorporated in England
& Wales. ShapeBlue Services India LLP is a company incorporated in India and is operated
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated
in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company
registered by The Republic of South Africa and is traded under license from Shape Blue Ltd.
ShapeBlue is a registered trademark.

Mime
View raw message