Mailing-List: contact cloudstack-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: cloudstack-dev@incubator.apache.org
Received-SPF: pass (nike.apache.org: domain of Alex.Huang@citrix.com
 designates 66.165.176.63 as permitted sender)
From: Alex Huang <Alex.Huang@citrix.com>
To: "cloudstack-dev@incubator.apache.org"
	<cloudstack-dev@incubator.apache.org>
Date: Tue, 5 Mar 2013 17:42:55 -0800
Subject: unit tests guidelines...
Thread-Topic: unit tests guidelines...
Thread-Index: Ac4Z+BYXZbqWB72dRF6KmP2+kOfGEA==
Message-ID: 
 <B1DF26ECC0458748AC97CECE2DA98D410131756498BD@SJCPMAILBOX01.citrite.net>
Accept-Language: en-US
Content-Language: en-US
acceptlanguage: en-US
Content-Type: multipart/alternative;
	boundary="_000_B1DF26ECC0458748AC97CECE2DA98D410131756498BDSJCPMAILBOX_"
MIME-Version: 1.0

--_000_B1DF26ECC0458748AC97CECE2DA98D410131756498BDSJCPMAILBOX_
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable

With the discussion in the BVT thread and other evidence from the check-ins=
, I think there are some confusion on when to and what to write in a unit t=
est.

Unit tests are a philosophy thing and I usually stay away from things like =
that.  If you are interested in writing the right type unit tests, here's m=
y $.02.

Unit tests are for

-          Guaranteeing the component is upholding its contract.

-          Illustrating how the component is to be used.

-          Mocking up failure scenarios

-          Explaining something that people might not understand if they lo=
ok inside your code.

When to write a unit test:

-          Write it at the component level because that's where the value i=
s.  You can test everything under the sun but basically code changes all of=
 the time.  What you really want

-          Write it for an interface.  For example, if you have AgentManage=
r and AgentManagerImpl, the methods you should test is in AgentManager.  An=
d a lot of this goes back to your design.  I have seen quite a bit of code =
that just adds all the methods from the class to the interface.  It's just =
something you do rather than something you think before doing.  That's just=
 wrong and it increases the number of unit tests you have to write.

These are usually what you need to test for when writing unit tests.

-          Errors in the incoming parameters...<no duh!>

-          Different positive paths for your code...<to state the obvious>.

-          The one that people don't seem to do is to inject results into t=
he components that the component being tested is dependent on.  This forces=
 component being tested to travel a different path.  Most people recognize =
incoming parameters causing a different path but does not recognize that re=
sults from the components being used can cause a different path.

How to recognize a good unit test?

-          The mock objects do not always return true or positive result.

-          The unit test sets something for the mock object, test the metho=
d; sets something else for the mock objects and then test the method again.=
  This means the tester is testing the component handling different returns=
 from the components it is using.

-          The unit test makes a call to the component, and checks the mock=
 object to see if certain things are called.  White box testing there.

-          The unit test has asserts for catching exceptions or negative re=
turns (negative paths are tested)

-          The unit test has comments that tell you what and why you're tes=
ting

-          The unit test asserts tells you why the assert should not fail

-          You won the blame game by saying look I have a test case for exa=
ctly that.

Misconceptions about unit tests.

-          You only write it when you write the component.  Actually, a goo=
d unit test is one that's progressively built up.  You found a bug, you wri=
te a test to make sure the bug is fixed.  If you've gotten to that stage, t=
hen you've pretty much reach guru status.

--Alex

--_000_B1DF26ECC0458748AC97CECE2DA98D410131756498BDSJCPMAILBOX_--