harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paulex Yang <paulex.y...@gmail.com>
Subject Re: [classlib] Testing
Date Fri, 28 Apr 2006 02:41:23 GMT
Mikhail Loenko wrote:
> Then how to run only independent tests?
>
> Such a run would make sense to validate the tests or
> if we switch to different RI version
>   
It should be easy for ant script to identify independent tests by 
package name convention, e.g, we have a keyword "API" for the 
independent tests (1), and  2 can  be easily excluded.

For the intended different behavior (4), test can be added to ant's 
exclude list (I don't expect there are too many such cases)

> Thanks,
> Mikhail
>
> 2006/4/28, Paulex Yang <paulex.yang@gmail.com>:
>   
>> Mikhail Loenko wrote:
>>     
>>> And how would you split 1,2,4,5  between these two folders?
>>>
>>>       
>> I thought it is obvious.
>> test 1,2,4 goes to src/test
>> test 5 goes to src/impltest (internl/injected/boot anything is fine)
>> and all implementation goes to src/main
>>
>> and they can be compiled to corresponding bin dir, i.e.
>> bin/test
>> bin/impltest
>> bin/main
>>
>> bin/main and bin/impltest goes to bootclasspath
>> bin/test goes to classpath
>>
>> Pls. correct me if I made mistake or missed sth.
>>     
>>> Thanks,
>>> Mikhail
>>>
>>> 2006/4/27, Paulex Yang <paulex.yang@gmail.com>:
>>>
>>>       
>>>> Mikhail Loenko wrote:
>>>>
>>>>         
>>>>> If it's separated, I'm OK with it
>>>>>
>>>>> Can you suggest dir names?
>>>>>
>>>>>
>>>>>           
>>>> Great!
>>>>
>>>> Either boot or injected(as Mark said) sounds good keyword to me.  so it
>>>> can be one of below:
>>>> 1. src/test.injected/
>>>> 2. src/test/injected
>>>> 3. src/test.boot/
>>>> 4. src/test/boot
>>>>
>>>> Mmm...seems injected is more self-descriptive,  I prefer option 1. So
>>>> the layout would be:
>>>> src/main
>>>> src/test
>>>> src/test.injected
>>>>
>>>> ideas?
>>>>
>>>>         
>>>>> Thanks,
>>>>> Mikhail
>>>>>
>>>>> 2006/4/27, Paulex Yang <paulex.yang@gmail.com>:
>>>>>
>>>>>
>>>>>           
>>>>>> Mikhail Loenko wrote:
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> The problem is the tests will be mixed like that:
>>>>>>>
>>>>>>> org.apache.harmony.security.asn1 -- tests in bootclasspath
>>>>>>> org.apache.harmony.security.cert  -- tests in bootclasspath
>>>>>>> org.apache.harmony.security.tests -- tests in classpath
>>>>>>> org.apache.harmony.security.x509 -- tests in bootclasspath
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> Good catch! :)
>>>>>>
>>>>>> It's not so hard for script to separate, if the package name doesn't
>>>>>> contains "test", it belongs to bootclasspath
>>>>>>
>>>>>> And if you concern about human reading, we can separate src directory
>>>>>> for them, i.e.
>>>>>> src/main/java
>>>>>> src/test/java
>>>>>> src/boottest/java or src/test/boot/java
>>>>>>
>>>>>> comments?
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> Thanks,
>>>>>>> Mikhail
>>>>>>>
>>>>>>> 2006/4/27, Paulex Yang <paulex.yang@gmail.com>:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> Mikhail Loenko wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> Paulex,
>>>>>>>>>
>>>>>>>>> we have at least 8 categories of tests:
>>>>>>>>>     running from classpath or bootclasspath
>>>>>>>>>     implementation specific or independent
>>>>>>>>>     testing org.apache.harmony.* or java.*
>>>>>>>>> Could you please list how all the tests will be named
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> Mikhail,
>>>>>>>>
>>>>>>>> OK, let me try. I'm not sure if all 8 categories are necessary,
>>>>>>>>
>>>>>>>> 1. [classpath], [independent], [java.*]
>>>>>>>>       o.a.h.<module name>.tests.api.<package name>,
e.g.,
>>>>>>>> o.a.h.luni.tests.api.java.util is test package for java.util
>>>>>>>>     I consider the module name is optional and can be removed,
so it can be
>>>>>>>>       o.a.h.tests.api.java.util for java.util
>>>>>>>>
>>>>>>>> 2. [classpath], [dependent], [o.a.h.*]
>>>>>>>>      o.a.h.<module name>.tests.<package name without
o.a.h...prefix>,
>>>>>>>> e.g., o.a.h.luni.tests.util is test package for o.a.h.luni.util
>>>>>>>>    Again, the module name is optional, but if module name
is not
>>>>>>>> prefix, it should be part of package name, so it is:
>>>>>>>>       o.a.h.tests.luni.util for o.a.h.luni.util
>>>>>>>>
>>>>>>>> 3. [classpath], [independent], [o.a.h.*]
>>>>>>>>    I don't expect these classes/tests exist.
>>>>>>>>
>>>>>>>> 4. [classpath], [dependent], [java.*]
>>>>>>>>    if my understanding is correct, this category means the
tests for
>>>>>>>> intended different behavior between Harmony and RI, I consider
these
>>>>>>>> tests should be put in same package with API tests, and separate
them by
>>>>>>>> running script instead of different package, because RI may
fix bug in
>>>>>>>> the revision later(e.g, Sun JDK 1.5.1 maybe?), and IIRC in
another
>>>>>>>> thread we have agreed that we always follow latest version
of RI, so it
>>>>>>>> will be much easier to change the different behavior tests
to API tests
>>>>>>>> if we use script to separate them, and vice versa.
>>>>>>>>
>>>>>>>> 5. [bootclasspath],
>>>>>>>>    which covers 4 categories you mentioned, the test cases
are in same
>>>>>>>> package with implementions
>>>>>>>>       e.g.
>>>>>>>>       java.util is test package for java.util
>>>>>>>>          and
>>>>>>>>       o.a.h.luni.util is test packag for o.a.h.luni.util
>>>>>>>>
>>>>>>>> As a summary, we'll have four kind of test packages:
>>>>>>>>
>>>>>>>> a. o.a.h.<module>.tests.api.java.***  for case 1,4
above
>>>>>>>> b. o.a.h.<module>.tests.***  for case 2 above
>>>>>>>> c. java.***, for case 5 above
>>>>>>>> d. o.a.h.***, for case 5 above
>>>>>>>>
>>>>>>>> comments?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> Thanks,
>>>>>>>>> Mikhail
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> 2006/4/27, Paulex Yang <paulex.yang@gmail.com>:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>> Mikhail Loenko wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>>> Geir,
>>>>>>>>>>>
>>>>>>>>>>> The problem is that no one yet suggested a consistent
solution
>>>>>>>>>>> that would fit for all the tests and would not
get into packages like
>>>>>>>>>>>
>>>>>>>>>>> org.apache.harmony.security.tests.org.apache.harmony.security.util
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Everybody seems to agree that SOME implementation
specific tests are
>>>>>>>>>>> in the same package as implementation.
>>>>>>>>>>>
>>>>>>>>>>> But if we have ALL the tests in the same package
as implementation,
>>>>>>>>>>> then we can't run e.g. tests against java.beans.Beans
from classpath, so
>>>>>>>>>>> we can't run regression test for HARMONY-358
>>>>>>>>>>>
>>>>>>>>>>> The question is where we put 'classpath'  test.
If we put them into
>>>>>>>>>>> o.a.h.module.tests.[package under test] we result
in that ugly package
>>>>>>>>>>> names like
>>>>>>>>>>> o.a.h.security.tests.o.a.h.security.util
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>> Mikhail,
>>>>>>>>>>
>>>>>>>>>> Why my prior proposal that o.a.h.security.tests.impl.util
is unacceptable?
>>>>>>>>>>
>>>>>>>>>> And I have no objection if "impl" considered redundant,
too.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Mikhail
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2006/4/27, Geir Magnusson Jr <geir@pobox.com>:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>>>> Geir Magnusson Jr wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>                         
>>>>>>>>>>>>> George Harley wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>                           
>>>>>>>>>>>>  >>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>                         
>>>>>>>>>>>>>> Of course, the text module has only
"implementation-independent tests
>>>>>>>>>>>>>> that designed to be run from classpath".
For modules that have got
>>>>>>>>>>>>>> implementation-specific tests then
I suppose we could use something
>>>>>>>>>>>>>> like "org.apache.harmony.[module].tests.impl.[package
under test]" or
>>>>>>>>>>>>>> "org.apache.harmony.[module].tests.internal.[package
under test]" etc.
>>>>>>>>>>>>>> I've got no preference.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>                             
>>>>>>>>>>>>> -1
>>>>>>>>>>>>>
>>>>>>>>>>>>> Put them in the same package as the class
being tested, please, if there
>>>>>>>>>>>>> is no problem being run that way.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>                           
>>>>>>>>>>>> Sorry to be so snippy.  Been a long day.
>>>>>>>>>>>>
>>>>>>>>>>>> Why would we want this convoluted package
name?  Why wouldn't we want to
>>>>>>>>>>>> let them be in the same package as the class
being tested since it's
>>>>>>>>>>>> implementation testing?
>>>>>>>>>>>>
>>>>>>>>>>>> geir
>>>>>>>>>>>>
>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>                         
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>> --
>>>>>>>>>> Paulex Yang
>>>>>>>>>> China Software Development Lab
>>>>>>>>>> IBM
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> --
>>>>>>>> Paulex Yang
>>>>>>>> China Software Development Lab
>>>>>>>> IBM
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> ---------------------------------------------------------------------
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> --
>>>>>> Paulex Yang
>>>>>> China Software Development Lab
>>>>>> IBM
>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> --
>>>> Paulex Yang
>>>> China Software Development Lab
>>>> IBM
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>>
>>>>         
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>>
>>>       
>> --
>> Paulex Yang
>> China Software Development Lab
>> IBM
>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>>     
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
>   


-- 
Paulex Yang
China Software Development Lab
IBM



---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message