harmony-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexei Fedotov" <alexei.fedo...@gmail.com>
Subject Re: [testing] make the tests structure comply with our testing conventions
Date Fri, 30 Nov 2007 13:43:57 GMT
Hello,
I prepared some patch for a testing conventions web page [1], but
noticed the following text at the bottom of the page

> javax.crypto.CipherTest   - Implementation independent bootclasspath test for javax.crypto.Cipher

So the patch needs to address this line as well to fight
inconsistence. My suggestion is to count the test as an implementation
dependent one since now just because it should be launched from
bootclasspath (and move it to impl/ subdirectory when the time
permits). Some of such tests may be later refactored to implementation
independent tests and placed to api/ package. Objections?

Thanks,

[1] http://issues.apache.org/jira/browse/HARMONY-5233

On Nov 28, 2007 3:11 PM, Alexei Fedotov <alexei.fedotov@gmail.com> wrote:
> Folks,
>
> I like Sean's suggestion - it removes a confusion while introduces
> minor changes. I also want to give you an option to vote against
> "boot" in favour of "java.injected". Personally, I prefer "boot"
> because it is shorter and doesn't contain dot which confuses
> xmlproperty mechanism.
>
> Please, participate. Otherwise Sean and I will dare to change the
> conventions in ths way.
>
> Thanks.
>
>
> On 11/28/07, Sean Qiu <sean.xx.qiu@gmail.com> wrote:
> > My suggestion is :
> >
> > <modulename>/src/test/api           // implementation-independent tests
> > <modulename>/src/test/impl         // platform-independent
> > implementation-dependent tests
> >        -common                             // implementation-dependent
> > tests
> >        -windows                              // windows-specific tests
> >        -unix                                    // linux-specific tests
> >        -boot                                    // tests which should be
> > injected into the bootclasspath
> >
> >  I totally agree with you that the src/test/api/common and src/test/api/java
> > are somehow duplicated in our page[1].
> > We should remove one of them, and boot is more appropriate and clear than
> > java.injected.
> >
> > We all know that a test is either api test or impl test. (except stress
> > test for special purpose , am i right? )
> > So, IMHO, it's more clear to divide them into two main parts first as we
> > have done before.
> > Then we can decide where they should be placed according to their special
> > usage.
> >
> > Any comments? Suggestions?
> >
> > [1] http://harmony.apache.org/subcomponents/classlibrary/testing.html
> >
> > 2007/11/28, Alexei Fedotov <alexei.fedotov@gmail.com>:
> > >
> > > Sean,
> > >
> > > > IMHO, the most important think is to make the modification in a minimun
> > > extent.
> > >
> > > That's good with me. So how do you think this might be implemented?
> > > Please, don't hesitate to correct my suggestion [1] or post something
> > > completely different from it.
> > >
> > > Thanks.
> > >
> > > [1]
> > > // implementation-independent tests
> > > <modulename>/src/test/api
> > > // platform-independent implementation-dependent tests
> > > <modulename>/src/test/impl
> > > // tests which should be injected into the bootclasspath
> > > <modulename>/src/test/boot
> > > // windows-specific tests
> > > <modulename>/src/test/windows
> > > // linux-specific tests
> > > <modulename>/src/test/linux
> > >
> > > On 11/28/07, Sean Qiu <sean.xx.qiu@gmail.com> wrote:
> > > > 2007/11/28, Alexei Fedotov <alexei.fedotov@gmail.com>:
> > > >
> > > > > Sean,
> > > > >
> > > > > Thanks for your letter. This is another step to achieve Harmony.
> > > > >
> > > > > You asked:
> > > > > > In this case, we will test them according to the behavior of
RI by
> > > > > > tradition. where should we put this api test to ?
> > > > >
> > > > > According to my last suggestion there are two options to place
> > > > > platform-dependent tests. If the test pretends to run on other
> > > > > implementations of Java, it should be placed at
> > > > >    <modulename>/src/test/api
> > > > >
> > > > > The platform dependency should be handled by conditional behavior
of
> > > > > test internals because its platform independent by design.
> > > > >
> > > > > If the test is intended to be for Harmony testing, it may be placed
at
> > > > >    <modulename>/src/test/<platform>
> > > >
> > > >
> > > > IMHO, the most important think is to make the modification in a minimun
> > > > extent.
> > > >
> > > >
> > > >
> > > >
> > > > > In return, I want to ask you direct question: what is your vision
of
> > > > > the ideal testing structure? I want somehow to resolve the
> > > > > inconsistency I've met.
> > > > >
> > > > > You wrote:
> > > > > > As mentioned by Tim earlier, the reason is based on their history
of
> > > > > some
> > > > > > being written early, some contributed, etc.
> > > >
> > > >
> > > > What i want to say here is that i think the other people all konw this
> > > > issue.
> > > > The reason that we havn't taken some actions is because it is
> > > > somehow important but less emergent.
> > > > And i agree with you that the standard should become consistent first.
> > > >
> > > >
> > > >
> > > > > This is not an excuse for having an inconsistent testing standard
on
> > > > > Harmony site. First, the standard should become consistent. Then
the
> > > > > tests should follow if the time permits.
> > > > >
> > > > > Finally, you conclude:
> > > > > > we need to reach a common consensus on the strcture for our
test
> > > before
> > > > > we start.
> > > >
> > > >
> > > > That's great,  and i think i can help for this if the communitiy reach
a
> > > > consensus.
> > > >
> > > > That's the main point I agree on. I would like to ask all people who
> > > > > might be affected by these standards to wake up and either agree,
or
> > > > > suggest consistent resolution of problems I get.
> > > > >
> > > > > Thank you for your interest.
> > > > >
> > > > > On 11/28/07, Sean Qiu <sean.xx.qiu@gmail.com> wrote:
> > > > > > 2007/11/28, Alexei Fedotov <alexei.fedotov@gmail.com>:
> > > > > >
> > > > > > > Hello,
> > > > > > > Thinking a bit further didn't help me to improve the
> > > understanding.
> > > > > > > How can we put the following examples into the testing
convention?
> > > > > > > <modulename>/src/test/api/java.injected
> > > > > > > <modulename>/src/test/api/windows
> > > > > > > <modulename>/src/test/api/linux
> > > > > >
> > > > > >
> > > > > > As mentioned by Tim earlier, the reason is based on their history
of
> > > > > some
> > > > > > being written early, some contributed, etc.
> > > > > >
> > > > > > IMHO, the confusion is caused by how we define *API* test.
> > > > > > According to the *spec*? Or according to the behavior of *RI*
> > > > > > The spec sounds fine in theory, but there do exist some
> > > functions  that
> > > > > are
> > > > > > platform dependent.
> > > > > > Such as the some functions of File, their behavior will vary
in
> > > > > different
> > > > > > platform in RI which is not mentioned in spec.
> > > > > > In this case, we will test them according to the behavior of
RI by
> > > > > > tridition.
> > > > > > where should we put this api test to ?
> > > > > >
> > > > > > I agree that it is time to tide them up now before their scale
> > > become
> > > > > out of
> > > > > > control.
> > > > > > IMHO, we need to reach a common consensus on the strcture for
our
> > > test
> > > > > > before we start.
> > > > > >
> > > > > >
> > > > > >
> > > > > > > How might platform-dependent or package-private tests fit
into api
> > > > > > > subcategory instead of impl? The Java specification which
is a
> > > > > > > backbone for implementation-independent tests is by design
> > > > > > > platform-independent and doesn't expose package-private
> > > information.
> > > > > > >
> > > > > > > Risking inhibiting a creative potential and an interest
of this
> > > thread
> > > > > > > I'd like to suggest the following solution which collapses
two
> > > > > > > descriptive directory layers into one:
> > > > > > >
> > > > > > > // implementation-independent tests
> > > > > > > <modulename>/src/test/api
> > > > > > > // platform-independent implementation-dependent tests
> > > > > > > <modulename>/src/test/impl
> > > > > > > // tests which should be injected into the bootclasspath
> > > > > > > <modulename>/src/test/boot
> > > > > > > // windows-specific tests
> > > > > > > <modulename>/src/test/windows
> > > > > > > // linux-specific tests
> > > > > > > <modulename>/src/test/linux
> > > > > > >
> > > > > > > What do you think?
> > > > > > >
> > > > > > > Thanks.
> > > > > > >
> > > > > > >
> > > > > > > On 11/27/07, Alexei Fedotov <alexei.fedotov@gmail.com>
wrote:
> > > > > > > > Thanks Tim, testing gurus,
> > > > > > > > I was looking into testing conventions [1] and got
the practical
> > > > > > > question:
> > > > > > > >
> > > > > > > > --- citation starts ---
> > > > > > > >
> > > > > > > > Tests are not separated by functionality under test,
for
> > > example,
> > > > > > > > tests against clone() methods are NOT separated from
tests
> > > against
> > > > > > > > equals() methods. Classpath tests are separated from
> > > bootclasspath
> > > > > > > > tests on a directory level:
> > > > > > > >
> > > > > > > > <modulename>/src/test/api/java - Classpath tests
> > > > > > > > <modulename>/src/test/api/java.injected - Bootclasspath
> > > testsFind
> > > > > more
> > > > > > > > details below.
> > > > > > > >
> > > > > > > > Some modules might have platform specific tests that
are in the
> > > case
> > > > > > > > separated on a directory level:
> > > > > > > >
> > > > > > > > <modulename>/src/test/api/common
> > > > > > > > <modulename>/src/test/api/windows
> > > > > > > > <modulename>/src/test/api/linux
> > > > > > > >
> > > > > > > > --- citation ends ---
> > > > > > > >
> > > > > > > > Imaging I have all kinds of tests, e.g. platform-specific,
> > > injected
> > > > > to
> > > > > > > > the boot classpath, etc. Where should I put MyTest.java
> > > > > > > >
> > > > > > > >   under src/test/api/common or under src/test/api/java?
> > > > > > > >
> > > > > > > > Or should I nest common under java? Or maybe we should
just
> > > replace
> > > > > > > > common with java, shouldn't we? I volunteer to prepare
a patch
> > > to
> > > > > the
> > > > > > > > web site when we agree on.
> > > > > > > >
> > > > > > > > [1]
> > > > > http://harmony.apache.org/subcomponents/classlibrary/testing.html
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > On 11/23/07, Tim Ellison <t.p.ellison@gmail.com>
wrote:
> > > > > > > > > Alexei Fedotov wrote:
> > > > > > > > > > Do I understand correctly that
> > > > > > > > > > working_classlib/modules/awt/src/test/api/java/
should be
> > > > > actually
> > > > > > > at
> > > > > > > > > > working_classlib/modules/awt/src/test/api/java.injected?
Is
> > > it
> > > > > > > > > > possible to use svn move instead of preparing
a monstrous
> > > patch?
> > > > > > > > >
> > > > > > > > > Just send along a script with the SVN commands
you want the
> > > > > committer
> > > > > > > to
> > > > > > > > > perform, that is how people have done it in the
past.
> > > > > > > > >
> > > > > > > > > > The reason why I'm asking is the following.
I need to add a
> > > test
> > > > > at
> > > > > > > > > > working_classlib/modules/awt/src/test/api/java/
and the
> > > tests
> > > > > which
> > > > > > > > > > are injected to a boot classpath are in
that place. I
> > > believe
> > > > > the
> > > > > > > > > > right way to solve the problem is to move
injected test
> > > cases to
> > > > > the
> > > > > > > > > > proper place first. What do you think?
> > > > > > > > >
> > > > > > > > > Sounds good.
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > > Tim
> > > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > --
> > > > > > > > With best regards,
> > > > > > > > Alexei,
> > > > > > > > ESSD, Intel
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > --
> > > > > > > With best regards,
> > > > > > > Alexei,
> > > > > > > ESSD, Intel
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Sean Qiu
> > > > > > http://xiaoxia.turendui.com
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > With best regards,
> > > > > Alexei,
> > > > > ESSD, Intel
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > Sean Qiu
> > > > http://xiaoxia.turendui.com
> > > >
> > >
> > >
> > > --
> > > With best regards,
> > > Alexei,
> > > ESSD, Intel
> > >
> >
> >
> >
> > --
> > Sean Qiu
> > http://xiaoxia.turendui.com
> >
>
>
> --
>
> With best regards,
> Alexei,
> ESSD, Intel
>



-- 
With best regards,
Alexei,
ESSD, Intel

Mime
View raw message