harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Beliaev <vladimir.k.beli...@gmail.com>
Subject Re: [drlvm][testing] build and run single regression test
Date Thu, 18 Jan 2007 11:49:31 GMT
Alexander,

let me join Alexey Varlamov to discussion since it was him who asked 
about presence of regression test for H2335, I believe he controls 
regression tests state and progress and can resolved mentioned issues 
pretty fast.

 >> 1. there should be opportunity to build regression test w/o rebuilding
 >> VM.
 >
 > It could be a usefull feature. Should go to TODO list.

I believe, it worth just get done because it is minor.

AFAIK one just need to define the ant target in
  <trunk>/working_vm/build/make/build.xml
which will call to "pre-reg-test" target in
  <trunk>/working_vm/build/make/targets/reg.test.xml

 >> 2. README should say what way the regression test can be built.
 >
 > Sorry, it is not clear.

I mean something like:
    BUILDING TEST #section name in README
    To build your regression test you need to execute the
    following commands:
    cd <trunk>/working_vm/build
    ./build.bat pre-reg-test # or better name for target

 > I tried to collect notes on infrastructure in [1] mail thread. You can
 > separate your notes on "features desired" and "readme notes" and post
 > them to the thread.

I believe, both notes are of the same kind. If regression test has a 
"development feature" then it just should be described in README.

Should I post set of my simple notes to [1]?

 >>    For example, H2335 regression test requires classes root dir and
 >> native library location are different. What way can I control this?
 >
 > It is done in such a way. README states it:

Right! I've missed this...

One more question to get answer from README:

6. what way the regression test may be excluded?

There are also set of questions for current status (state) of 
regressions tests I got after I run these tests with command:
    cd <trunk>/working_vm/build
    ./build.bat reg.test # builds & runs all regression tests

7. Regression test H1694 does not answer naming conversion. There is no 
java source ending with Test in this directory. (I saw the comments says 
  this test does not pass anyway). Let's get this test fixed to answer 
naming convention.

8. sometimes the tests run reports are "coupled" (no empty string and 
string with "run-test" between tests run reports) - difficult to see in 
log whether or not test is passed. E.g.:

run-test:
     [junit] Running org....h0000.DirectByteBufferTest
     [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 0 sec
     [junit] Test org....h0000.DirectByteBufferTest FAILED
     [junit] Running org.....h1800.ExecTest
     [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 0.27 sec

run-test:
     [junit] Running org.....h2151.VMInit1Test
     [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 1.832sec

   Can this be controlled? And if "yes", then it would bee nice to get 
reports separated, please.

thanks
Vladimir Beliaev

Alexander Kleymenov wrote:
> Hi Vladimir,
> 
> On 1/18/07, Vladimir Beliaev <vladimir.k.beliaev@gmail.com> wrote:
>> Hello,
>>
>> While working on http://issues.apache.org/jira/browse/HARMONY-2335 I got
>> the note - regression test is required. Fine - I've start looking at
>> <trunk>/working_vm/src/test/regression/README.txt and got the following
>> points related to regression test development which would be nice to get
>> reflected:
>>
>> 1. there should be opportunity to build regression test w/o rebuilding 
>> VM.
> 
> It could be a usefull feature. Should go to TODO list.
> 
>> 2. README should say what way the regression test can be built.
> 
> Sorry, it is not clear.
> Do you want the words "java sources are compiled by java's compiller,
> c by c's, cpp by cpp's, jasmin by jasmin's" to be there?
> 
>> 3. there should be opportunity to run single regression test.
>> 4. README should say what way a single regression test can be run.
> 
> Another TODO for reg testing infrastructure.
> 
>>
>> I can file JIRA on this if required. If the questions are asked already
>> somewhere in html (site) documentation, then the link in this README is
>> just fine.
> 
> I tried to collect notes on infrastructure in [1] mail thread. You can
> separate your notes on "features desired" and "readme notes" and post
> them to the thread.
> 
>> There is one more specific question:
>>
>> 5. would be helpful to understand (from README) the layout of built
>> regression tests.
>>
>>    For example, H2335 regression test requires classes root dir and
>> native library location are different. What way can I control this?
> 
> It is done in such a way. README states it:
> 
>    ${reg.test.class.path} - the path to where java and
>        java assembler sources were compiled
> 
>    ${reg.test.2launch.native.path} - the path to where native
>        sources were compiled
> 
>> Still this question is of the low priority. To be honest, the simplest
>> regression test for H2335 does not require a test native part. And
>> probably such type of regression test is really rare.
>>
>> Thanks
>>
>> Vladimir Beliaev
> 
> Thank you,
> Alexander
> 
> [1] [drlvm][testing] Regression Testing Infrastructure proposal
> 


Mime
View raw message