harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mikhail Loenko <mloe...@gmail.com>
Subject Re: [testing] code for exotic configurations
Date Fri, 27 Jan 2006 13:33:35 GMT
Hi George,

On 1/27/06, George Harley <george.c.harley@googlemail.com> wrote:
> Sure. And it should be the role of the test framework to inform users
> about which tests got run, passed, failed, got skipped etc.

see below

> > It would not disturb most of the people because the test will pass in 'bad'
> > environment. But those, who know about these tests will sometimes grep
> > logs to validate configuration.
> >
> Why use logs to tell us information that the unit test framework can already
> provide ? Who wants to have to grep through log files when the test runner
> can provide us with what we need ?

Sorry but I did not quite understand. I do not see anything related to
the 'skipped' status in the junit.framework.TestResult

If there is such a status then it would be excellent, it is the option #1 in the
first mail in the thread

Thanks,
Mikhail

>
> > Thanks,
> > Mikhail
> >
> >
> >
> >> Alternatively, they could be
> >> included as part of a general test suite but be purposely skipped over at
> >> test execution time using a
> >> test exclusion list understood by the test runner.
> >>
> >>
> >> Best regards,
> >> George
> >> ________________________________________
> >> George C. Harley
> >>
> >>
> >>
> >>
> >>
> >> Tim Ellison <t.p.ellison@gmail.com>
> >> 27/01/2006 08:53
> >> Please respond to
> >> harmony-dev@incubator.apache.org
> >>
> >>
> >> To
> >> harmony-dev@incubator.apache.org
> >> cc
> >>
> >> Subject
> >> Re: [testing] code for exotic configurations
> >>
> >>
> >>
> >>
> >>
> >>
> >> Anton Avtamonov wrote:
> >>
> >>>> Note that I could create my own provider and test with it, but what
I
> >>>>
> >> would
> >>
> >>>> really want is to test how my EncryptedPrivateKeyInfo works with
> >>>> AlgorithmParameters from real provider as well as how my other classes
> >>>>
> >> work
> >>
> >>>> with real implementations of crypto Engines.
> >>>>
> >>>> Thanks,
> >>>> Mikhail.
> >>>>
> >>>>
> >>> Hi Mikhail,
> >>> There are 'system' and 'unit' tests. Traditionally, unit tests are of
> >>> developer-level. Each unit test is intended to test just a limited
> >>> piece of functionality separately from other sub-systems (test for one
> >>> fucntion, test for one class, etc). Such tests must create a desired
> >>> environment over the testing fucntionality and run the scenario in the
> >>> predefined conditions. Unit tests usually able to cover all scenarios
> >>> (execution paths) for the tested parts of fucntionality.
> >>>
> >>> What are you talking about looks like 'system' testing. Such tests
> >>> usually run on the real environment and test the most often scenarious
> >>> (the reduntant set, all scenarios usually cannot be covered). Such
> >>> testing is not concentrated on the particular fucntionality, but
> >>> covers the work of the whole system.
> >>> A sample is: "run some demo application on some particular platform,
> >>> with some particular providers installed and perform some operations".
> >>>
> >>> I think currently we should focus on 'unit' test approach since it is
> >>> more applicable during the development (so my advise is to revert your
> >>> tests to install 'test' providers with the desired behavior as George
> >>> proposed).
> >>> However we should think about 'system' scenarios which can be run on
> >>> the later stage and act as 'verification' of proper work of the entire
> >>> system.
> >>>
> >> I agree with all this.  The unit tests are one style of test for
> >> establishing the correctness of the code.  As you point out the unit
> >> tests typically require a well-defined environment in which to run, and
> >> it becomes a judgment-call as to whether a particular test's
> >> environmental requirements are 'reasonable' or not.
> >>
> >> For example, you can reasonably expect all developers to have an
> >> environment to run unit tests that has enough RAM and a writable disk
> >> etc. such that if those things do not exist the tests will simply fail.
> >>  However, you may decide it is unreasonable to expect the environment to
> >> include a populated LDAP server, or a carefully configured RMI server.
> >> If you were to call that environment unreasonable then testing JNDI and
> >> RMI would likely involve mock objects etc. to get good unit tests.
> >>
> >> Of course, as you point out, once you are passing the unit tests you
> >> also need the 'system' tests to ensure the code works in a real
> >> environment.  Usage scenarios based on the bigger system are good, as is
> >> running the bigger system's test suite on our runtime.
> >>
> >> Regards,
> >> Tim
> >>
> >>
> >>
> >>> --
> >>> Anton Avtamonov,
> >>> Intel Middleware Products Division
> >>>
> >>>
> >> --
> >>
> >> Tim Ellison (t.p.ellison@gmail.com)
> >> IBM Java technology centre, UK.
> >>
> >>
> >>
> >>
> >
> >
>
>

Mime
View raw message