harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Fedotov" <alexei.fedo...@gmail.com>
Subject Re: [testing] make the tests structure comply with our testing conventions
Date Fri, 30 Nov 2007 17:09:50 GMT
Hello,
A short update: someone already refactored crypto module in a way Sean
and I suggested, so the only thing I had to do with the inconsistency
was updating the example with description of current situation. This
means the patch for [1] is ready.

[1] http://issues.apache.org/jira/browse/HARMONY-5233

On Nov 30, 2007 4:43 PM, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> Hello,
> I prepared some patch for a testing conventions web page [1], but
> noticed the following text at the bottom of the page
>
> > javax.crypto.CipherTest   - Implementation independent bootclasspath test for javax.crypto.Cipher
>
> So the patch needs to address this line as well to fight
> inconsistence. My suggestion is to count the test as an implementation
> dependent one since now just because it should be launched from
> bootclasspath (and move it to impl/ subdirectory when the time
> permits). Some of such tests may be later refactored to implementation
> independent tests and placed to api/ package. Objections?
>
> Thanks,
>
> [1] http://issues.apache.org/jira/browse/HARMONY-5233
>
>
> On Nov 28, 2007 3:11 PM, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> > Folks,
> >
> > I like Sean's suggestion - it removes a confusion while introduces
> > minor changes. I also want to give you an option to vote against
> > "boot" in favour of "java.injected". Personally, I prefer "boot"
> > because it is shorter and doesn't contain dot which confuses
> > xmlproperty mechanism.
> >
> > Please, participate. Otherwise Sean and I will dare to change the
> > conventions in ths way.
> >
> > Thanks.
> >
> >
> > On 11/28/07, Sean Qiu <sean.xx.qiu@gmail.com> wrote:
> > > My suggestion is :
> > >
> > > <modulename>/src/test/api           // implementation-independent tests
> > > <modulename>/src/test/impl         // platform-independent
> > > implementation-dependent tests
> > >        -common                             // implementation-dependent
> > > tests
> > >        -windows                              // windows-specific tests
> > >        -unix                                    // linux-specific tests
> > >        -boot                                    // tests which should be
> > > injected into the bootclasspath
> > >
> > >  I totally agree with you that the src/test/api/common and src/test/api/java
> > > are somehow duplicated in our page[1].
> > > We should remove one of them, and boot is more appropriate and clear than
> > > java.injected.
> > >
> > > We all know that a test is either api test or impl test. (except stress
> > > test for special purpose , am i right? )
> > > So, IMHO, it's more clear to divide them into two main parts first as we
> > > have done before.
> > > Then we can decide where they should be placed according to their special
> > > usage.
> > >
> > > Any comments? Suggestions?
> > >
> > > [1] http://harmony.apache.org/subcomponents/classlibrary/testing.html
> > >
> > > 2007/11/28, Alexei Fedotov <alexei.fedotov@gmail.com>:
> > > >
> > > > Sean,
> > > >
> > > > > IMHO, the most important think is to make the modification in a minimun
> > > > extent.
> > > >
> > > > That's good with me. So how do you think this might be implemented?
> > > > Please, don't hesitate to correct my suggestion [1] or post something
> > > > completely different from it.
> > > >
> > > > Thanks.
> > > >
> > > > [1]
> > > > // implementation-independent tests
> > > > <modulename>/src/test/api
> > > > // platform-independent implementation-dependent tests
> > > > <modulename>/src/test/impl
> > > > // tests which should be injected into the bootclasspath
> > > > <modulename>/src/test/boot
> > > > // windows-specific tests
> > > > <modulename>/src/test/windows
> > > > // linux-specific tests
> > > > <modulename>/src/test/linux
> > > >
> > > > On 11/28/07, Sean Qiu <sean.xx.qiu@gmail.com> wrote:
> > > > > 2007/11/28, Alexei Fedotov <alexei.fedotov@gmail.com>:
> > > > >
> > > > > > Sean,
> > > > > >
> > > > > > Thanks for your letter. This is another step to achieve Harmony.
> > > > > >
> > > > > > You asked:
> > > > > > > In this case, we will test them according to the behavior
of RI by
> > > > > > > tradition. where should we put this api test to ?
> > > > > >
> > > > > > According to my last suggestion there are two options to place
> > > > > > platform-dependent tests. If the test pretends to run on other
> > > > > > implementations of Java, it should be placed at
> > > > > >    <modulename>/src/test/api
> > > > > >
> > > > > > The platform dependency should be handled by conditional behavior
of
> > > > > > test internals because its platform independent by design.
> > > > > >
> > > > > > If the test is intended to be for Harmony testing, it may be
placed at
> > > > > >    <modulename>/src/test/<platform>
> > > > >
> > > > >
> > > > > IMHO, the most important think is to make the modification in a minimun
> > > > > extent.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > > In return, I want to ask you direct question: what is your vision
of
> > > > > > the ideal testing structure? I want somehow to resolve the
> > > > > > inconsistency I've met.
> > > > > >
> > > > > > You wrote:
> > > > > > > As mentioned by Tim earlier, the reason is based on their
history of
> > > > > > some
> > > > > > > being written early, some contributed, etc.
> > > > >
> > > > >
> > > > > What i want to say here is that i think the other people all konw
this
> > > > > issue.
> > > > > The reason that we havn't taken some actions is because it is
> > > > > somehow important but less emergent.
> > > > > And i agree with you that the standard should become consistent first.
> > > > >
> > > > >
> > > > >
> > > > > > This is not an excuse for having an inconsistent testing standard
on
> > > > > > Harmony site. First, the standard should become consistent.
Then the
> > > > > > tests should follow if the time permits.
> > > > > >
> > > > > > Finally, you conclude:
> > > > > > > we need to reach a common consensus on the strcture for
our test
> > > > before
> > > > > > we start.
> > > > >
> > > > >
> > > > > That's great,  and i think i can help for this if the communitiy
reach a
> > > > > consensus.
> > > > >
> > > > > That's the main point I agree on. I would like to ask all people
who
> > > > > > might be affected by these standards to wake up and either agree,
or
> > > > > > suggest consistent resolution of problems I get.
> > > > > >
> > > > > > Thank you for your interest.
> > > > > >
> > > > > > On 11/28/07, Sean Qiu <sean.xx.qiu@gmail.com> wrote:
> > > > > > > 2007/11/28, Alexei Fedotov <alexei.fedotov@gmail.com>:
> > > > > > >
> > > > > > > > Hello,
> > > > > > > > Thinking a bit further didn't help me to improve the
> > > > understanding.
> > > > > > > > How can we put the following examples into the testing
convention?
> > > > > > > > <modulename>/src/test/api/java.injected
> > > > > > > > <modulename>/src/test/api/windows
> > > > > > > > <modulename>/src/test/api/linux
> > > > > > >
> > > > > > >
> > > > > > > As mentioned by Tim earlier, the reason is based on their
history of
> > > > > > some
> > > > > > > being written early, some contributed, etc.
> > > > > > >
> > > > > > > IMHO, the confusion is caused by how we define *API* test.
> > > > > > > According to the *spec*? Or according to the behavior of
*RI*
> > > > > > > The spec sounds fine in theory, but there do exist some
> > > > functions  that
> > > > > > are
> > > > > > > platform dependent.
> > > > > > > Such as the some functions of File, their behavior will
vary in
> > > > > > different
> > > > > > > platform in RI which is not mentioned in spec.
> > > > > > > In this case, we will test them according to the behavior
of RI by
> > > > > > > tridition.
> > > > > > > where should we put this api test to ?
> > > > > > >
> > > > > > > I agree that it is time to tide them up now before their
scale
> > > > become
> > > > > > out of
> > > > > > > control.
> > > > > > > IMHO, we need to reach a common consensus on the strcture
for our
> > > > test
> > > > > > > before we start.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > > How might platform-dependent or package-private tests
fit into api
> > > > > > > > subcategory instead of impl? The Java specification
which is a
> > > > > > > > backbone for implementation-independent tests is by
design
> > > > > > > > platform-independent and doesn't expose package-private
> > > > information.
> > > > > > > >
> > > > > > > > Risking inhibiting a creative potential and an interest
of this
> > > > thread
> > > > > > > > I'd like to suggest the following solution which collapses
two
> > > > > > > > descriptive directory layers into one:
> > > > > > > >
> > > > > > > > // implementation-independent tests
> > > > > > > > <modulename>/src/test/api
> > > > > > > > // platform-independent implementation-dependent tests
> > > > > > > > <modulename>/src/test/impl
> > > > > > > > // tests which should be injected into the bootclasspath
> > > > > > > > <modulename>/src/test/boot
> > > > > > > > // windows-specific tests
> > > > > > > > <modulename>/src/test/windows
> > > > > > > > // linux-specific tests
> > > > > > > > <modulename>/src/test/linux
> > > > > > > >
> > > > > > > > What do you think?
> > > > > > > >
> > > > > > > > Thanks.
> > > > > > > >
> > > > > > > >
> > > > > > > > On 11/27/07, Alexei Fedotov <alexei.fedotov@gmail.com>
wrote:
> > > > > > > > > Thanks Tim, testing gurus,
> > > > > > > > > I was looking into testing conventions [1] and
got the practical
> > > > > > > > question:
> > > > > > > > >
> > > > > > > > > --- citation starts ---
> > > > > > > > >
> > > > > > > > > Tests are not separated by functionality under
test, for
> > > > example,
> > > > > > > > > tests against clone() methods are NOT separated
from tests
> > > > against
> > > > > > > > > equals() methods. Classpath tests are separated
from
> > > > bootclasspath
> > > > > > > > > tests on a directory level:
> > > > > > > > >
> > > > > > > > > <modulename>/src/test/api/java - Classpath
tests
> > > > > > > > > <modulename>/src/test/api/java.injected
- Bootclasspath
> > > > testsFind
> > > > > > more
> > > > > > > > > details below.
> > > > > > > > >
> > > > > > > > > Some modules might have platform specific tests
that are in the
> > > > case
> > > > > > > > > separated on a directory level:
> > > > > > > > >
> > > > > > > > > <modulename>/src/test/api/common
> > > > > > > > > <modulename>/src/test/api/windows
> > > > > > > > > <modulename>/src/test/api/linux
> > > > > > > > >
> > > > > > > > > --- citation ends ---
> > > > > > > > >
> > > > > > > > > Imaging I have all kinds of tests, e.g. platform-specific,
> > > > injected
> > > > > > to
> > > > > > > > > the boot classpath, etc. Where should I put MyTest.java
> > > > > > > > >
> > > > > > > > >   under src/test/api/common or under src/test/api/java?
> > > > > > > > >
> > > > > > > > > Or should I nest common under java? Or maybe
we should just
> > > > replace
> > > > > > > > > common with java, shouldn't we? I volunteer to
prepare a patch
> > > > to
> > > > > > the
> > > > > > > > > web site when we agree on.
> > > > > > > > >
> > > > > > > > > [1]
> > > > > > http://harmony.apache.org/subcomponents/classlibrary/testing.html
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > On 11/23/07, Tim Ellison <t.p.ellison@gmail.com>
wrote:
> > > > > > > > > > Alexei Fedotov wrote:
> > > > > > > > > > > Do I understand correctly that
> > > > > > > > > > > working_classlib/modules/awt/src/test/api/java/
should be
> > > > > > actually
> > > > > > > > at
> > > > > > > > > > > working_classlib/modules/awt/src/test/api/java.injected?
Is
> > > > it
> > > > > > > > > > > possible to use svn move instead of
preparing a monstrous
> > > > patch?
> > > > > > > > > >
> > > > > > > > > > Just send along a script with the SVN commands
you want the
> > > > > > committer
> > > > > > > > to
> > > > > > > > > > perform, that is how people have done it
in the past.
> > > > > > > > > >
> > > > > > > > > > > The reason why I'm asking is the following.
I need to add a
> > > > test
> > > > > > at
> > > > > > > > > > > working_classlib/modules/awt/src/test/api/java/
and the
> > > > tests
> > > > > > which
> > > > > > > > > > > are injected to a boot classpath are
in that place. I
> > > > believe
> > > > > > the
> > > > > > > > > > > right way to solve the problem is to
move injected test
> > > > cases to
> > > > > > the
> > > > > > > > > > > proper place first. What do you think?
> > > > > > > > > >
> > > > > > > > > > Sounds good.
> > > > > > > > > >
> > > > > > > > > > Regards,
> > > > > > > > > > Tim
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > --
> > > > > > > > > With best regards,
> > > > > > > > > Alexei,
> > > > > > > > > ESSD, Intel
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > With best regards,
> > > > > > > > Alexei,
> > > > > > > > ESSD, Intel
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > Sean Qiu
> > > > > > > http://xiaoxia.turendui.com
> > > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > With best regards,
> > > > > > Alexei,
> > > > > > ESSD, Intel
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Sean Qiu
> > > > > http://xiaoxia.turendui.com
> > > > >
> > > >
> > > >
> > > > --
> > > > With best regards,
> > > > Alexei,
> > > > ESSD, Intel
> > > >
> > >
> > >
> > >
> > > --
> > > Sean Qiu
> > > http://xiaoxia.turendui.com
> > >
> >
> >
> > --
> >
> > With best regards,
> > Alexei,
> > ESSD, Intel
> >
>
>
>
> --
>
> With best regards,
> Alexei,
> ESSD, Intel
>



-- 
With best regards,
Alexei,
ESSD, Intel

Mime
View raw message