harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paulex Yang <paulex.y...@gmail.com>
Subject Re: [classlib] Testing
Date Fri, 28 Apr 2006 06:42:39 GMT
Mikhail Loenko wrote:
> Hi Paulex
>
> 2006/4/28, Paulex Yang <paulex.yang@gmail.com>:
>   
>> Mikhail Loenko wrote:
>>     
>>> Then how to run only independent tests?
>>>
>>> Such a run would make sense to validate the tests or
>>> if we switch to different RI version
>>>
>>>       
>> It should be easy for ant script to identify independent tests by
>> package name convention, e.g, we have a keyword "API" for the
>> independent tests (1), and  2 can  be easily excluded.
>>
>> For the intended different behavior (4), test can be added to ant's
>> exclude list (I don't expect there are too many such cases)
>>     
>
> Did you count such tests? As I remember we have a lof of them in security.
>
>   
Lots of ? Ooops...Seems I didn't understand the situation clearly (I 
knew I must miss sth. ;-) )
> If these tests are in the same package and in the same directory
> then we will have something like
>
> org.apache.harmony.security.tests.api.java.security.CodeSignerTest1 - impl
> specific test methods
>
> org.apache.harmony.security.tests.api.java.security.CodeSignerTest2 - impl
> independent test methods
>   
If there are many similar cases, I agree that it is 
unacceptable(although I think the test case can be named more 
meaningfully than xxxTest1/xxxTest2), because I prefer 1 test for 1 
class in general .  It will be perfect if we can introduce some 
mechanism to control test execution in method level, but I have no good 
idea on this so far.

So do you have any better ideas on this issue?
> How do you like it?
>
> Thanks,
> Mikhail
>
>   
>>> Thanks,
>>> Mikhail
>>>
>>> 2006/4/28, Paulex Yang <paulex.yang@gmail.com>:
>>>
>>>       
>>>> Mikhail Loenko wrote:
>>>>
>>>>         
>>>>> And how would you split 1,2,4,5  between these two folders?
>>>>>
>>>>>
>>>>>           
>>>> I thought it is obvious.
>>>> test 1,2,4 goes to src/test
>>>> test 5 goes to src/impltest (internl/injected/boot anything is fine)
>>>> and all implementation goes to src/main
>>>>
>>>> and they can be compiled to corresponding bin dir, i.e.
>>>> bin/test
>>>> bin/impltest
>>>> bin/main
>>>>
>>>> bin/main and bin/impltest goes to bootclasspath
>>>> bin/test goes to classpath
>>>>
>>>> Pls. correct me if I made mistake or missed sth.
>>>>
>>>>         
>>>>> Thanks,
>>>>> Mikhail
>>>>>
>>>>> 2006/4/27, Paulex Yang <paulex.yang@gmail.com>:
>>>>>
>>>>>
>>>>>           
>>>>>> Mikhail Loenko wrote:
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> If it's separated, I'm OK with it
>>>>>>>
>>>>>>> Can you suggest dir names?
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> Great!
>>>>>>
>>>>>> Either boot or injected(as Mark said) sounds good keyword to me.
 so it
>>>>>> can be one of below:
>>>>>> 1. src/test.injected/
>>>>>> 2. src/test/injected
>>>>>> 3. src/test.boot/
>>>>>> 4. src/test/boot
>>>>>>
>>>>>> Mmm...seems injected is more self-descriptive,  I prefer option 1.
So
>>>>>> the layout would be:
>>>>>> src/main
>>>>>> src/test
>>>>>> src/test.injected
>>>>>>
>>>>>> ideas?
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> Thanks,
>>>>>>> Mikhail
>>>>>>>
>>>>>>> 2006/4/27, Paulex Yang <paulex.yang@gmail.com>:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>>> Mikhail Loenko wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> The problem is the tests will be mixed like that:
>>>>>>>>>
>>>>>>>>> org.apache.harmony.security.asn1 -- tests in bootclasspath
>>>>>>>>> org.apache.harmony.security.cert  -- tests in bootclasspath
>>>>>>>>> org.apache.harmony.security.tests -- tests in classpath
>>>>>>>>> org.apache.harmony.security.x509 -- tests in bootclasspath
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> Good catch! :)
>>>>>>>>
>>>>>>>> It's not so hard for script to separate, if the package name
doesn't
>>>>>>>> contains "test", it belongs to bootclasspath
>>>>>>>>
>>>>>>>> And if you concern about human reading, we can separate src
directory
>>>>>>>> for them, i.e.
>>>>>>>> src/main/java
>>>>>>>> src/test/java
>>>>>>>> src/boottest/java or src/test/boot/java
>>>>>>>>
>>>>>>>> comments?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>>>> Thanks,
>>>>>>>>> Mikhail
>>>>>>>>>
>>>>>>>>> 2006/4/27, Paulex Yang <paulex.yang@gmail.com>:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>>>> Mikhail Loenko wrote:
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>>> Paulex,
>>>>>>>>>>>
>>>>>>>>>>> we have at least 8 categories of tests:
>>>>>>>>>>>     running from classpath or bootclasspath
>>>>>>>>>>>     implementation specific or independent
>>>>>>>>>>>     testing org.apache.harmony.* or java.*
>>>>>>>>>>> Could you please list how all the tests will
be named
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>> Mikhail,
>>>>>>>>>>
>>>>>>>>>> OK, let me try. I'm not sure if all 8 categories
are necessary,
>>>>>>>>>>
>>>>>>>>>> 1. [classpath], [independent], [java.*]
>>>>>>>>>>       o.a.h.<module name>.tests.api.<package
name>, e.g.,
>>>>>>>>>> o.a.h.luni.tests.api.java.util is test package for
java.util
>>>>>>>>>>     I consider the module name is optional and can
be removed, so it can be
>>>>>>>>>>       o.a.h.tests.api.java.util for java.util
>>>>>>>>>>
>>>>>>>>>> 2. [classpath], [dependent], [o.a.h.*]
>>>>>>>>>>      o.a.h.<module name>.tests.<package
name without o.a.h...prefix>,
>>>>>>>>>> e.g., o.a.h.luni.tests.util is test package for o.a.h.luni.util
>>>>>>>>>>    Again, the module name is optional, but if module
name is not
>>>>>>>>>> prefix, it should be part of package name, so it
is:
>>>>>>>>>>       o.a.h.tests.luni.util for o.a.h.luni.util
>>>>>>>>>>
>>>>>>>>>> 3. [classpath], [independent], [o.a.h.*]
>>>>>>>>>>    I don't expect these classes/tests exist.
>>>>>>>>>>
>>>>>>>>>> 4. [classpath], [dependent], [java.*]
>>>>>>>>>>    if my understanding is correct, this category
means the tests for
>>>>>>>>>> intended different behavior between Harmony and RI,
I consider these
>>>>>>>>>> tests should be put in same package with API tests,
and separate them by
>>>>>>>>>> running script instead of different package, because
RI may fix bug in
>>>>>>>>>> the revision later(e.g, Sun JDK 1.5.1 maybe?), and
IIRC in another
>>>>>>>>>> thread we have agreed that we always follow latest
version of RI, so it
>>>>>>>>>> will be much easier to change the different behavior
tests to API tests
>>>>>>>>>> if we use script to separate them, and vice versa.
>>>>>>>>>>
>>>>>>>>>> 5. [bootclasspath],
>>>>>>>>>>    which covers 4 categories you mentioned, the test
cases are in same
>>>>>>>>>> package with implementions
>>>>>>>>>>       e.g.
>>>>>>>>>>       java.util is test package for java.util
>>>>>>>>>>          and
>>>>>>>>>>       o.a.h.luni.util is test packag for o.a.h.luni.util
>>>>>>>>>>
>>>>>>>>>> As a summary, we'll have four kind of test packages:
>>>>>>>>>>
>>>>>>>>>> a. o.a.h.<module>.tests.api.java.***  for case
1,4 above
>>>>>>>>>> b. o.a.h.<module>.tests.***  for case 2 above
>>>>>>>>>> c. java.***, for case 5 above
>>>>>>>>>> d. o.a.h.***, for case 5 above
>>>>>>>>>>
>>>>>>>>>> comments?
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>>>> Thanks,
>>>>>>>>>>> Mikhail
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> 2006/4/27, Paulex Yang <paulex.yang@gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>>>> Mikhail Loenko wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>                         
>>>>>>>>>>>>> Geir,
>>>>>>>>>>>>>
>>>>>>>>>>>>> The problem is that no one yet suggested
a consistent solution
>>>>>>>>>>>>> that would fit for all the tests and
would not get into packages like
>>>>>>>>>>>>>
>>>>>>>>>>>>> org.apache.harmony.security.tests.org.apache.harmony.security.util
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> Everybody seems to agree that SOME implementation
specific tests are
>>>>>>>>>>>>> in the same package as implementation.
>>>>>>>>>>>>>
>>>>>>>>>>>>> But if we have ALL the tests in the same
package as implementation,
>>>>>>>>>>>>> then we can't run e.g. tests against
java.beans.Beans from classpath, so
>>>>>>>>>>>>> we can't run regression test for HARMONY-358
>>>>>>>>>>>>>
>>>>>>>>>>>>> The question is where we put 'classpath'
 test. If we put them into
>>>>>>>>>>>>> o.a.h.module.tests.[package under test]
we result in that ugly package
>>>>>>>>>>>>> names like
>>>>>>>>>>>>> o.a.h.security.tests.o.a.h.security.util
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>                           
>>>>>>>>>>>> Mikhail,
>>>>>>>>>>>>
>>>>>>>>>>>> Why my prior proposal that o.a.h.security.tests.impl.util
is unacceptable?
>>>>>>>>>>>>
>>>>>>>>>>>> And I have no objection if "impl" considered
redundant, too.
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>                         
>>>>>>>>>>>>> Thanks,
>>>>>>>>>>>>> Mikhail
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2006/4/27, Geir Magnusson Jr <geir@pobox.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>                           
>>>>>>>>>>>>>> Geir Magnusson Jr wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>                             
>>>>>>>>>>>>>>> George Harley wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                             
 
>>>>>>>>>>>>>>  >>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>                             
>>>>>>>>>>>>>>>> Of course, the text module
has only "implementation-independent tests
>>>>>>>>>>>>>>>> that designed to be run from
classpath". For modules that have got
>>>>>>>>>>>>>>>> implementation-specific tests
then I suppose we could use something
>>>>>>>>>>>>>>>> like "org.apache.harmony.[module].tests.impl.[package
under test]" or
>>>>>>>>>>>>>>>> "org.apache.harmony.[module].tests.internal.[package
under test]" etc.
>>>>>>>>>>>>>>>> I've got no preference.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>                         
       
>>>>>>>>>>>>>>> -1
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Put them in the same package
as the class being tested, please, if there
>>>>>>>>>>>>>>> is no problem being run that
way.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>                             
 
>>>>>>>>>>>>>> Sorry to be so snippy.  Been a long
day.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Why would we want this convoluted
package name?  Why wouldn't we want to
>>>>>>>>>>>>>> let them be in the same package as
the class being tested since it's
>>>>>>>>>>>>>> implementation testing?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> geir
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>>>>> For additional commands, e-mail:
harmony-dev-help@incubator.apache.org
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>                             
>>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>                           
>>>>>>>>>>>> --
>>>>>>>>>>>> Paulex Yang
>>>>>>>>>>>> China Software Development Lab
>>>>>>>>>>>> IBM
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>                         
>>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>                       
>>>>>>>>>> --
>>>>>>>>>> Paulex Yang
>>>>>>>>>> China Software Development Lab
>>>>>>>>>> IBM
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>                     
>>>>>>>>> ---------------------------------------------------------------------
>>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>>> --
>>>>>>>> Paulex Yang
>>>>>>>> China Software Development Lab
>>>>>>>> IBM
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>                 
>>>>>>> ---------------------------------------------------------------------
>>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> --
>>>>>> Paulex Yang
>>>>>> China Software Development Lab
>>>>>> IBM
>>>>>>
>>>>>>
>>>>>>
>>>>>> ---------------------------------------------------------------------
>>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>> ---------------------------------------------------------------------
>>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>           
>>>> --
>>>> Paulex Yang
>>>> China Software Development Lab
>>>> IBM
>>>>
>>>>
>>>>
>>>> ---------------------------------------------------------------------
>>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>>
>>>>
>>>>
>>>>         
>>> ---------------------------------------------------------------------
>>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>>
>>>
>>>
>>>       
>> --
>> Paulex Yang
>> China Software Development Lab
>> IBM
>>
>>
>>
>> ---------------------------------------------------------------------
>> Terms of use : http://incubator.apache.org/harmony/mailing.html
>> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
>> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>>
>>
>>     
>
> ---------------------------------------------------------------------
> Terms of use : http://incubator.apache.org/harmony/mailing.html
> To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
> For additional commands, e-mail: harmony-dev-help@incubator.apache.org
>
>
>   


-- 
Paulex Yang
China Software Development Lab
IBM



---------------------------------------------------------------------
Terms of use : http://incubator.apache.org/harmony/mailing.html
To unsubscribe, e-mail: harmony-dev-unsubscribe@incubator.apache.org
For additional commands, e-mail: harmony-dev-help@incubator.apache.org


Mime
View raw message